Commit Graph

78 Commits

Author SHA1 Message Date
Nick Sweeting
30c60eef76 much better tests and add page ui 2025-12-29 04:02:11 -08:00
Nick Sweeting
f4e7820533 use full dotted paths for all archivebox imports, add migrations and more fixes 2025-12-29 00:47:08 -08:00
Nick Sweeting
f0aa19fa7d wip 2025-12-28 17:51:54 -08:00
Nick Sweeting
bd265c0083 rename extractor to plugin everywhere 2025-12-28 04:43:15 -08:00
Nick Sweeting
50e527ec65 way better plugin hooks system wip 2025-12-28 03:39:59 -08:00
Claude
b632894bc9 Update views, API, and exports for new ArchiveResult output fields
Replace old `output` field with new fields across the codebase:
- output_str: Human-readable output summary
- output_json: Structured metadata (optional)
- output_files: Dict of output files with metadata
- output_size: Total size in bytes
- output_mimetypes: CSV of file mimetypes

Files updated:
- api/v1_core.py: Update MinimalArchiveResultSchema to expose new fields
- api/v1_core.py: Update ArchiveResultFilterSchema to search output_str
- cli/archivebox_extract.py: Use output_str in CLI output
- core/admin_archiveresults.py: Update admin fields, search, and fieldsets
- core/admin_archiveresults.py: Fix output_html variable name bug in output_summary
- misc/jsonl.py: Update archiveresult_to_jsonl() to include new fields
- plugins/extractor_utils.py: Update ExtractorResult helper class

The embed_path() method already uses output_files and output_str,
so snapshot detail page and template tags work correctly.
2025-12-27 20:28:22 +00:00
Claude
13be196fd7 Merge remote-tracking branch 'origin/dev' into claude/improve-test-suite-xm6Bh
# Conflicts:
#	pyproject.toml
2025-12-27 02:27:51 +00:00
Nick Sweeting
e2cbcd17f6 more tests and migrations fixes 2025-12-26 18:22:48 -08:00
Claude
ae2ab5b273 Add Python 3.13 support with uuid7 backport compatibility
- Create uuid_compat.py module that provides uuid7 for Python <3.14
  using uuid_extensions package, and native uuid.uuid7 for Python 3.14+
- Update all model files and migrations to use archivebox.uuid_compat
- Add uuid7 conditional dependency in pyproject.toml for Python <3.14
- Update requires-python to >=3.13 (from >=3.14)
- Update GitHub workflows, lock_pkgs.sh to use Python 3.13
- Update tool configs (ruff, pyright, uv) for Python 3.13

This enables running ArchiveBox on Python 3.13 while maintaining
forward compatibility with Python 3.14's native uuid7 support.
2025-12-27 01:07:30 +00:00
Nick Sweeting
bb53228ebf remove Seed model in favor of Crawl as template 2025-12-25 01:52:41 -08:00
Nick Sweeting
866f993f26 logging and admin ui improvements 2025-12-25 01:10:41 -08:00
Nick Sweeting
d95f0dc186 remove huey 2025-12-24 23:40:18 -08:00
Nick Sweeting
6c769d831c wip 2 2025-12-24 21:46:14 -08:00
Nick Sweeting
1915333b81 wip major changes 2025-12-24 20:10:38 -08:00
Nick Sweeting
c1335fed37 Remove ABID system and KVTag model - use UUIDv7 IDs exclusively
This commit completes the simplification of the ID system by:

- Removing the ABID (ArchiveBox ID) system entirely
- Removing the base_models/abid.py file
- Removing KVTag model in favor of the existing Tag model in core/models.py
- Simplifying all models to use standard UUIDv7 primary keys
- Removing ABID-related admin functionality
- Cleaning up commented-out ABID code from views and statemachines
- Deleting migration files for ABID field removal (no longer needed)

All models now use simple UUIDv7 ids via `id = models.UUIDField(primary_key=True, default=uuid7)`

Note: Old migrations containing ABID references are preserved for database
migration history compatibility.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-24 06:13:49 -08:00
Nick Sweeting
c374d7695e allow getting crawl from API as rss feed 2024-12-03 02:13:45 -08:00
Nick Sweeting
328eb98a38 move main funcs into cli files and switch to using click for CLI 2024-11-19 00:18:51 -08:00
Nick Sweeting
569081a9eb rename abid_utils to base_models 2024-11-18 19:40:05 -08:00
Nick Sweeting
65afd405b1 merge seeds and crawls apps 2024-11-18 19:23:14 -08:00
Nick Sweeting
e469c5a344 merge queues and actors apps into new workers app 2024-11-18 18:52:48 -08:00
Nick Sweeting
eeb2671e4d API improvements 2024-11-18 04:27:38 -08:00
Nick Sweeting
c7bd9449d5 better jobs dashboard with faster refresh 2024-11-18 04:27:38 -08:00
Nick Sweeting
eb53145e4e working state machine flow yay 2024-11-18 04:27:38 -08:00
Nick Sweeting
1e3ce67834 fix API and CLU calls 2024-11-18 04:27:38 -08:00
Nick Sweeting
b852442efc add crawls app back to django admin 2024-11-18 04:27:37 -08:00
Nick Sweeting
8f8fbbb7a2 API fixes and add actors endpoints 2024-11-17 20:09:06 -08:00
Nick Sweeting
43514da0d0 add crawl and seed endpoints to REST API 2024-11-16 02:45:11 -08:00
Ben Muthalaly
4213d7dc27 Fix API crash 2024-10-26 01:53:49 -05:00
Nick Sweeting
c0b7887fd7 fix admin registration using abx hooks 2024-10-14 17:38:38 -07:00
Nick Sweeting
f75ae805f8 comment out Crawl api methods temporarily 2024-10-14 15:41:58 -07:00
Nick Sweeting
518c46b4ab fix circular import and show log of plugins loading on startup 2024-10-14 15:35:03 -07:00
Nick Sweeting
cf1ea8f80f improve config loading of TMP_DIR, LIB_DIR, move to separate files 2024-10-07 23:45:11 -07:00
Nick Sweeting
73e69ccb8b fixes for docs generation 2024-10-04 19:16:46 -07:00
Nick Sweeting
d21bc86075 finish migrating almost all config to new system 2024-09-30 23:21:34 -07:00
Nick Sweeting
363a499289 move util.py into misc folder 2024-09-30 17:25:15 -07:00
Nick Sweeting
3e5b6ddeae move config into dedicated global app 2024-09-30 15:59:05 -07:00
Nick Sweeting
d8a9dca0f6 use constants in more places 2024-09-26 02:38:45 -07:00
Nick Sweeting
e99260feb2 fix rich logging issues 2024-09-24 21:17:07 -07:00
Nick Sweeting
44669fab73 add BaseHook concept to underlie all Plugin hooks 2024-09-05 03:36:18 -07:00
Nick Sweeting
ed5357cec9 add migrations for datetime field renames 2024-09-04 23:44:13 -07:00
Nick Sweeting
0c236b034b allow accessing admin change pages by .abid as well as .id 2024-09-04 23:43:07 -07:00
Nick Sweeting
cbf2a8fdc3 rename datetime fields to _at, massively improve ABID generation safety and determinism 2024-09-04 23:42:36 -07:00
Nick Sweeting
68a39b7392 remove .old_id entirely and make ABID generation only happen once on initial save 2024-09-04 16:40:15 -07:00
Nick Sweeting
01094ecb03 fix REST API CSRF and auth handling 2024-09-03 14:16:44 -07:00
Nick Sweeting
c1c55d6da7 fix API token_auth and CSRF setup 2024-09-03 01:21:13 -07:00
Nick Sweeting
f1579bfdcd add NODE_BIN_PATH to dynamic CONFIG and make ANSI an AttrDict 2024-09-03 00:19:18 -07:00
Nick Sweeting
dd05ad04fa disable cookie auth in API because csrf=False 2024-09-02 01:13:19 -07:00
Nick Sweeting
1a03db2b1d fix api auth endpoint checking wrong user field 2024-08-22 18:28:23 -07:00
Nick Sweeting
57e48b432c show redacted API tokens in django admin 2024-08-20 19:25:16 -07:00
Nick Sweeting
ff619fd31f add api migrations 2024-08-20 18:31:40 -07:00