website-downloader/README.md
liushuang f645d616f8 init
2024-08-13 11:45:11 +08:00

29 lines
1.1 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

## Website Downloader
Download the complete source code of any website (including all assets)
## Description
Website downloader works with `wget` and `archiver` to download all websites assets and compress then sends it back to the user through socket channel
**wget params the being used**
`wget --mirror --convert-links --adjust-extension --page-requisites
--no-parent http://example.org`
**Explanation of the various flags:**
- --mirror Makes (among other things) the download recursive.
- --convert-links convert all the links (also to stuff like CSS stylesheets) to relative, so it will be suitable for offline viewing.
- --adjust-extension Adds suitable extensions to filenames (html or css) depending on their content-type.
- --page-requisites Download things like CSS style-sheets and images required to properly display the page offline.
- --no-parent When recursing do not ascend to the parent directory. It useful for restricting the download to only a portion of the site
## Run
- `$ npm install`
- `$ npm start`
- `http://localhost:3008/`