alex/ArchiveBox
1
0
Fork 0
mirror of https://github.com/ArchiveBox/ArchiveBox.git synced 2026-04-03 14:27:55 +10:00
Code Issues Packages Projects Releases Wiki Activity

merge dev

Browse Source
This commit is contained in:
Nick Sweeting
2024-11-02 17:34:33 -07:00
parent dbe5c0bc07 27f26fdde8
commit 9b24fe7390
19 changed files with 632 additions and 275 deletions
Download Patch File Download Diff File Expand all files Collapse all files

198
.github/ISSUE_TEMPLATE/1-bug_report.yml vendored Normal file
View File

@@ -0,0 +1,198 @@
name: 🐞 Bug report
description: Report a bug or error you encountered in ArchiveBox
title: "Bug: ..."
assignees:
- pirate
type: 'Bug'
body:
- type: markdown
attributes:
value: |
*Please note:* it is normal to see errors occasionally for some extractors on some URLs (not every extractor will work on every type of page).
Please report archiving errors if you are seeing them *consistently across many URLs* or if they are *preventing you from using ArchiveBox*.
- type: textarea
id: description
attributes:
label: Provide a screenshot and describe the bug
description: |
Attach a screenshot and describe what the issue is, what you expected to happen, and if relevant, the *URLs you were trying to archive*.
placeholder: |
Got a bunch of 'singlefile was unable to archive this page' errors when trying to archive URLs from this site: https://example.com/xyz ...
I also tried to archive the same URLs using `singlefile` directly and some of them worked but not all of them. etc. ...
validations:
required: true
- type: textarea
id: steps_to_reproduce
attributes:
label: Steps to reproduce
description: Please include the exact steps you took to trigger the issue.
render: markdown
placeholder: |
1. Started ArchiveBox by running: `docker run -v $PWD:/data -p 8000:8000 archivebox/archivebox:latest`
2. Went to the https://127.0.0.1:8000/add/ page in Google Chrome
3. Typed 'https://example.com/xyz' into the 'Add URL' input field
4. Clicked the 'Add+' button
5. Got a 500 error and saw the errors below in terminal
validations:
required: true
- type: textarea
id: logs
attributes:
label: Logs or errors
description: "Paste any terminal output, logs, or errors (check `data/logs/errors.log` as well)."
placeholder: |
╭─────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ [2024-11-02 19:54:28] ArchiveBox v0.8.6rc0: archivebox add https://example.com#1234567 │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯
[+] [2024-11-02 19:54:29] Adding 1 links to index (crawl depth=0)...
> Saved verbatim input to sources/1730577269-import.txt
> Parsed 1 URLs from input (Generic TXT)
...
render: shell
validations:
required: false
- type: textarea
id: version
attributes:
label: ArchiveBox Version
description: |
**REQUIRED:** Run the `archivebox version` command inside your collection dir and paste the *full output* here (*not just the version number*).
For Docker Compose run: `docker compose run archivebox version`
For plain Docker run: `docker run -v $PWD:/data archivebox/archivebox version`
render: shell
placeholder: |
0.8.6
ArchiveBox v0.8.6rc0 COMMIT_HASH=721427a BUILD_TIME=2024-10-21 12:57:02 1729515422
IN_DOCKER=False IN_QEMU=False ARCH=arm64 OS=Darwin PLATFORM=macOS-15.1-arm64-arm-64bit PYTHON=Cpython (venv)
EUID=502:20 UID=502:20 PUID=502:20 FS_UID=502:20 FS_PERMS=644 FS_ATOMIC=True FS_REMOTE=False
DEBUG=False IS_TTY=True SUDO=False ID=dfa11485:aa78ad45 SEARCH_BACKEND=ripgrep LDAP=False
Binary Dependencies:
√ python 3.11.9 venv_pip ~/.venv/bin/python
√ django 5.1.2 venv_pip ~/.venv/lib/python3.11/site-packages/django/__init__.py
√ sqlite 2.6.0 venv_pip ~/.venv/lib/python3.11/site-packages/django/db/backends/sqlite3/base.py
√ pip 24.3.1 venv_pip ~/.venv/bin/pip
...
validations:
required: true
- type: dropdown
id: install_method
validations:
required: true
attributes:
label: How did you install the version of ArchiveBox you are using?
multiple: false
options:
- pip
- apt
- brew
- nix
- Docker (or other container system like podman/LXC/Kubernetes or TrueNAS/Cloudron/YunoHost/etc.)
- Other
- type: dropdown
id: operating_system
validations:
required: true
attributes:
label: What operating system are you running on?
description: |
Please note we are *unable to provide support for Windows users* unless you are using [Docker on Windows](https://github.com/ArchiveBox/archivebox#:~:text=windows%20without%20docker).
multiple: false
options:
- Linux (Ubuntu/Debian/Arch/Alpine/etc.)
- macOS (including Docker on macOS)
- BSD (FreeBSD/OpenBSD/NetBSD/etc.)
- Windows (including WSL, WSL2, Docker Desktop on Windows)
- Other
- type: checkboxes
id: filesystem
attributes:
label: What type of drive are you using to store your ArchiveBox data?
description: Are you using a [remote filesystem](https://github.com/ArchiveBox/ArchiveBox/wiki/Setting-Up-Storage#supported-remote-filesystems) or FUSE mount for `data/` or `data/archive` (e.g. NFS/SMB/CIFS/etc. or FUSE/RClone/S3/B2/OneDrive/etc.)?
options:
- label: "`data/` is on a local SSD or NVMe drive"
required: false
- label: "`data/` is on a spinning hard drive or external USB drive"
required: false
- label: "`data/` is on a network mount (e.g. NFS/SMB/CIFS/etc.)"
required: false
- label: "`data/` is on a FUSE mount (e.g. SSHFS/RClone/S3/B2/OneDrive, etc.)"
required: false
- type: textarea
id: docker_compose_yml
attributes:
label: Docker Compose Configuration
description: "If using Docker Compose, please share your full `docker-compose.yml` file. If using plain Docker, paste the `docker run ...` command you use."
placeholder: |
services:
archivebox:
image: archivebox/archivebox:latest
ports:
- 8000:8000
volumes:
- ./data:/data
environment:
- ADMIN_USERNAME=admin
- ADMIN_PASSWORD=****<redact any passwords>****
- ALLOWED_HOSTS=*
- CSRF_TRUSTED_ORIGINS=https://archivebox.example.com
- PUBLIC_INDEX=True
- PUBLIC_SNAPSHOTS=True
- PUBLIC_ADD_VIEW=False
...
archivebox_scheduler:
image: archivebox/archivebox:latest
command: schedule --foreground --update --every=day
environment:
...
...
render: shell
validations:
required: false
- type: textarea
id: configuration
attributes:
label: ArchiveBox Configuration
description: "Please share your full `data/ArchiveBox.conf` file here."
render: shell
placeholder: |
[SERVER_CONFIG]
SECRET_KEY = "*********<redact any secrets/passwords>************"
WGET_RESTRICT_FILE_NAMES=windows
USE_SYSTEM_WGET=true
CHECK_SSL_VALIDITY=false
...
validations:
required: false
- type: markdown
attributes:
value: |
---
We strive to answer issues as quickly as possible, it usually takes us *about a ~week* to respond.
Make sure your `data/` is [**fully backed up**](https://github.com/ArchiveBox/ArchiveBox/wiki/Usage#disk-layout) before trying anything suggested here, **we are not responsible for data loss**.
In the meantime please consider:
- 💰 [Donating to support ArchiveBox open-source](https://github.com/ArchiveBox/ArchiveBox/wiki/Donations)
- 👨‍✈️ [Hiring us for corporate deployments](https://docs.monadical.com/s/archivebox-consulting-services) with professional support, custom feature development, and help with CAPTCHAs/rate-limits
- 🔍 [Searching the Documentation](https://docs.archivebox.io/) for answers to common questions
- 📚 Reading the [Troubleshooting Guide](https://github.com/ArchiveBox/ArchiveBox/wiki)
- ✨ Testing out a newer [`BETA` release](https://github.com/ArchiveBox/ArchiveBox/releases) (issues are often already fixed in our latest `BETA` releases)

127
.github/ISSUE_TEMPLATE/2-feature_request.yml vendored Normal file
View File

@@ -0,0 +1,127 @@
name: 💡 Feature or enhancement request
description: Suggest an idea or improvement for this project
title: "Feature Request: ..."
assignees:
- pirate
type: 'Enhancement'
labels: 'status: idea phase'
body:
- type: dropdown
id: suggestion_type
validations:
required: true
attributes:
label: "What type of suggestion are you making?"
multiple: false
options:
- New extractor / type of content to save
- Proposing a new feature
- Modification of existing behavior
- Web UI or UX design improvement
- type: textarea
id: current_problem
attributes:
label: "What is the problem that your feature request solves?"
description: |
Describe the problem that your feature request solves, feel free to include any screenshots or examples.
placeholder: |
e.g. I need to be able to archive spanish and french subtitle files from a particular movie site https://example.com/somevideos that's going down soon.
validations:
required: true
- type: textarea
id: proposed_solution
attributes:
label: "What is your proposed solution?"
description: |
Describe the ideal specific solution you'd want, and whether it fits into any broader scope of changes.
placeholder: |
e.g. I specifically need a new archive method to look for multilingual subtitle files related to pages.
The bigger picture solution is the ability for custom user scripts to be run in a puppeteer context during archiving.
validations:
required: true
- type: textarea
id: workarounds_tried
attributes:
label: "What hacks or alternative solutions have you tried to solve the problem?"
description: |
A clear and concise description of any alternative solutions, workarounds, or other software you've considered using to fix the problem.
placeholder: |
e.g. I wait for archivebox to finish archiving the page, then I manually run `yt-dlp --subs <URL>` inside
the `data/archive/<timestamp>/` directory to download the subtitle files and add them to the snapshot folder.
validations:
required: true
- type: textarea
id: version
attributes:
label: What version of ArchiveBox are you currently using?
description: |
We need to know what version of ArchiveBox and what OS you're currently using in order to contextualize your feature request.
Sometimes we've already fixed the issues in newer BETA versions, sometimes features already exist but may not be available in specific environments/versions.
Run the `archivebox version` command inside your current collection dir and paste the *full output* here (*not just the version number*).
For Docker Compose run: `docker compose run archivebox version`
For plain Docker run: `docker run -v $PWD:/data archivebox/archivebox version`
render: shell
placeholder: |
0.8.6
ArchiveBox v0.8.6rc0 COMMIT_HASH=721427a BUILD_TIME=2024-10-21 12:57:02 1729515422
IN_DOCKER=False IN_QEMU=False ARCH=arm64 OS=Darwin PLATFORM=macOS-15.1-arm64-arm-64bit PYTHON=Cpython (venv)
EUID=502:20 UID=502:20 PUID=502:20 FS_UID=502:20 FS_PERMS=644 FS_ATOMIC=True FS_REMOTE=False
DEBUG=False IS_TTY=True SUDO=False ID=dfa11485:aa78ad45 SEARCH_BACKEND=ripgrep LDAP=False
Binary Dependencies:
√ python 3.11.9 venv_pip ~/.venv/bin/python
√ django 5.1.2 venv_pip ~/.venv/lib/python3.11/site-packages/django/__init__.py
√ sqlite 2.6.0 venv_pip ~/.venv/lib/python3.11/site-packages/django/db/backends/sqlite3/base.py
√ pip 24.3.1 venv_pip ~/.venv/bin/pip
...
validations:
required: true
- type: checkboxes
id: priority
attributes:
label: "How badly do you want this new feature?"
options:
- label: "It's an urgent deal-breaker, I can't live without it"
required: false
- label: "It's important to add it in the near-mid term future"
required: false
- label: "It would be nice to have eventually"
required: false
- label: "I'm willing to [work on a PR](https://github.com/ArchiveBox/ArchiveBox#archivebox-development) to develop this myself"
required: false
- label: "I have [donated money](https://github.com/ArchiveBox/ArchiveBox/wiki/Donations) to go towards fixing this issue"
required: false
- type: checkboxes
id: satisfaction_survey
attributes:
label: Mini Survey
description: How do you like ArchiveBox so far?
options:
- label: "I like ArchiveBox so far / would recommend it to a friend"
required: false
- label: "I've had a lot of difficulty getting ArchiveBox set up"
required: false
- label: "I would pay $10/mo for a hosted version of ArchiveBox if it had this feature"
required: false
- type: markdown
attributes:
value: |
---
We strive to answer issues as quickly as possible, it usually takes us *about a ~week* to respond.
Make sure your `data/` is [**fully backed up**](https://github.com/ArchiveBox/ArchiveBox/wiki/Usage#disk-layout) before trying any workarounds or BETAs suggested here, **we are not responsible for data loss**.
In the meantime please consider:
- 💰 [Donating to support ArchiveBox open-source](https://github.com/ArchiveBox/ArchiveBox/wiki/Donations)
- 📊 [Hiring us for corporate deployments](https://docs.monadical.com/s/archivebox-consulting-services) with professional support, custom feature development, and help with CAPTCHAs/rate-limits
- 🔍 [Searching the Documentation](https://docs.archivebox.io/) for answers to common questions
- ✨ Testing out a newer [`BETA` release](https://github.com/ArchiveBox/ArchiveBox/releases) (issues are often already fixed in our latest `BETA` releases)

52
.github/ISSUE_TEMPLATE/3-documentation_change.yml vendored Normal file
View File

@@ -0,0 +1,52 @@
name: 📑 Documentation improvement
description: Submit an idea or correction for the Wiki documentation
title: "Documentation: ..."
labels: 'touches: docs'
type: 'Enhancement'
assignees:
- pirate
body:
- type: markdown
attributes:
value: |
If you prefer, you can submit a [Pull Request](https://github.com/ArchiveBox/docs) on https://github.com/ArchiveBox/docs to edit the docs directly instead.
- type: input
id: page_url
validations:
required: true
attributes:
label: "What is the URL of the page you'd like to see improved?"
placeholder: e.g. https://github.com/ArchiveBox/docs/wiki/Install
- type: input
id: section_title
validations:
required: true
attributes:
label: "What is the title of the relevant section?"
placeholder: e.g. Option B. Automatic Setup Script
- type: textarea
id: suggested_edit
attributes:
label: "What is the suggested edit?"
placeholder: |
e.g. Please document how to run the automatic setup script for ArchiveBox on TempleOS.
Attach images, screenshots, code snippets, etc. anything you think would help.
validations:
required: true
- type: markdown
attributes:
value: |
---
We strive to address issues as quickly as possible, it usually takes us *about a ~week* to respond.
In the meantime please consider:
- 💰 [Donating to support ArchiveBox open-source](https://github.com/ArchiveBox/ArchiveBox/wiki/Donations)
- 👨‍✈️ [Hiring us for corporate deployments](https://docs.monadical.com/s/archivebox-consulting-services) with professional support, custom feature development, and help with CAPTCHAs/rate-limits
- 🔍 [Checking out the new ReadTheDocs Documentation](https://docs.archivebox.io/)
- ✨ Helping us test a newer [`BETA` release](https://github.com/ArchiveBox/ArchiveBox/releases)

46
.github/ISSUE_TEMPLATE/bug_report.md vendored
View File

@@ -1,46 +0,0 @@
---
name: 🐞 Bug report
about: Create a report to help us improve
title: 'Bug: ...'
labels: 'bug'
assignees: ''
---
<!--
Please fill out the following information,
feel free to delete sections if they're not applicable
or if long issue templates annoy you.
(the only required section is the version information)
-->
#### Describe the bug
<!--
A description of what the bug is,
what you expected to happen,
and any relevant context about issue.
-->
#### Steps to reproduce
<!--
For example:
1. Ran ArchiveBox with the following config '...'
2. Saw this output during archiving '....'
3. UI didn't show the thing I was expecting '....'
-->
#### Screenshots or log output
<!--
If applicable, post any relevant screenshots or copy/pasted terminal output from ArchiveBox.
If you're reporting a parsing / importing error, **you must paste a copy of your redacted import file here**.
-->
#### ArchiveBox version
<!-- Run the `archivebox version` command locally then copy paste the result here: -->
```logs
replace this line with the *full*, unshortened output of running `archivebox version`
```
<!-- Tickets without full version info will closed until it is provided,
we need the full output here to help you solve your issue -->

8
.github/ISSUE_TEMPLATE/config.yml vendored Normal file
View File

@@ -0,0 +1,8 @@
blank_issues_enabled: false
contact_links:
- name: 💬 Chat with us on Zulip
url: https://zulip.archivebox.io
about: "Join us on our Zulip forum to chat with the developers and other users (it's similar to Discord but self-hosted)."
- name: 💁‍♂️ Hire us to provide archiving for your organization
url: https://docs.monadical.com/s/archivebox-consulting-services
about: "We provide hosting, develoment, and support, including on-prem/cloud setup w/ SSO & storage, CAPTCHA/rate-limiting avoidance, etc."

17
.github/ISSUE_TEMPLATE/documentation_change.md vendored
View File

@@ -1,17 +0,0 @@
---
name: 📑 Documentation change
about: Submit a suggestion for the Wiki documentation
title: 'Documentation: Improvement request ...'
labels: ''
assignees: ''
---
<!-- If you perfer, you can make a PR to https://github.com/ArchiveBox/docs instead of opening an issue -->
## Wiki Page URL
<!-- e.g. https://github.com/pirate/ArchiveBox/wiki/Configuration#use_color -->
## Suggested Edit
<!-- e.g. Please add more example usages, or please fix `xyz` typo to be `abc`. -->

50
.github/ISSUE_TEMPLATE/feature_request.md vendored
View File

@@ -1,50 +0,0 @@
---
name: 💡 Feature request
about: Suggest an idea for this project
title: 'Feature Request: ...'
labels: 'changes: behavior,status: idea phase'
assignees: ''
---
<!--
Please fill out the following information,
feel free to delete sections if they're not applicable
or if long issue templates annoy you :)
-->
## Type
- [ ] General question or discussion
- [ ] Propose a brand new feature
- [ ] Request modification of existing behavior or design
## What is the problem that your feature request solves
<!--
e.g. I need to be able to archive spanish and french subtitle files
from a particular <example.com> movie site that's going down soon.
-->
## Describe the ideal specific solution you'd want, and whether it fits into any broader scope of changes
<!--
e.g. I specifically need a new archive method to look for multilingual subtitle files related to pages.
The bigger picture solution is the ability for custom user scripts to be run in a puppeteer context during archiving.
-->
## What hacks or alternative solutions have you tried to solve the problem?
<!--
A clear and concise description of any alternative solutions,
workarounds, or other software you've considered using to fix the problem.
-->
## How badly do you want this new feature?
- [ ] It's an urgent deal-breaker, I can't live without it
- [ ] It's important to add it in the near-mid term future
- [ ] It would be nice to have eventually
---
- [ ] I'm willing to contribute [dev time](https://github.com/ArchiveBox/ArchiveBox#archivebox-development) / [money](https://github.com/sponsors/pirate) to fix this issue
- [ ] I like ArchiveBox so far / would recommend it to a friend
- [ ] I've had a lot of difficulty getting ArchiveBox set up

9
.github/ISSUE_TEMPLATE/question_or_discussion.md vendored
View File

@@ -1,9 +0,0 @@
---
name: 💬 Question, discussion, or support request
about: Start a discussion or ask a question about ArchiveBox
title: 'Question: ...'
labels: ''
assignees: ''
---