# WTG **Archived project; service retired.** The system collected publications, ran OCR and classification, then showed them to an operator for review. The service, its data and backups have been removed from the VPS. Anonymized screenshots, diagrams and measurements are kept here. Source version: `21841487f52489d8bafaac379b9cc08199937661` Code-map documents: June 2026; retained data history: June–July. Archiving and deletion: 7 September 2026. [Project notes](https://portfolio.komaroff-dev.ru/en/projects/wtg/) · [Online documentation](https://portfolio.komaroff-dev.ru/en/projects/wtg/docs/) [Source repository](https://github.com/komaroffsergei/wtg-s) ## Task Each publication goes through several processing steps. The interface needed to show its source, OCR output, classification and the operator's decision. Errors and queues have separate screens. Collecting publications and manually reviewing processed results. ## Work done - Collection, queues, processing and the operator review screen. - Admin interface, data models and code map. - Saved anonymized screenshots and aggregates before removal. The service, production data and backups were deleted. ## Stack | Technology | Role | | --- | --- | | Python / FastAPI / Jinja | Operations admin interface, JSON API and orchestration. | | PostgreSQL / SQLAlchemy / Alembic | Publications, jobs, decision versions, materials and migrations. | | PaddleOCR | Separate OCR sidecar and worker. | | Docker / Nginx / systemd | Former deployment of seven services and background jobs. | ## Architecture and decisions - Collection and long-running processing are separate jobs with leases and heartbeats. - OCR is separate from the web service, with its own queue, statuses and attempts. - Source publications, intermediate decisions and final materials have different roles. - The archive contains only anonymized images, documentation and aggregates; the working database, sessions and backups were not retained. ### Observed architecture before deletion ![Observed architecture before deletion](https://portfolio.komaroff-dev.ru/evidence/wtg/diagrams/architecture.svg) [Mermaid source](https://portfolio.komaroff-dev.ru/evidence/wtg/diagrams/architecture.mmd) ### Data model before deletion ![Data model before deletion](https://portfolio.komaroff-dev.ru/evidence/wtg/diagrams/data-model.svg) [Mermaid source](https://portfolio.komaroff-dev.ru/evidence/wtg/diagrams/data-model.mmd) ### Former deployment: seven containers ![Former deployment: seven containers](https://portfolio.komaroff-dev.ru/evidence/wtg/diagrams/deployment.svg) [Mermaid source](https://portfolio.komaroff-dev.ru/evidence/wtg/diagrams/deployment.mmd) ### Processing sequence before deletion ![Processing sequence before deletion](https://portfolio.komaroff-dev.ru/evidence/wtg/diagrams/processing-sequence.svg) [Mermaid source](https://portfolio.komaroff-dev.ru/evidence/wtg/diagrams/processing-sequence.mmd) ### Historical target architecture from source code; not a plan to restart WTG ![Historical target architecture from source code; not a plan to restart WTG](https://portfolio.komaroff-dev.ru/evidence/wtg/diagrams/target-processing-layers.svg) [Mermaid source](https://portfolio.komaroff-dev.ru/evidence/wtg/diagrams/target-processing-layers.mmd) ## Code map | Feature | Screen / API | Module | Data / verification | | --- | --- | --- | --- | | Authorization | accounts | services/auth.py, vk_user_token.py | auth_accounts, account_communities; browser/API states are considered separately | | Job creation and execution | schedules, processing | services/admin_control.py, cli/admin_worker.py | admin_jobs, admin_job_step_logs; claim, heartbeat, retry, stale cleanup | | API scheduling | schedules, group-pool | services/api_posts_schedule.py | api_schedule_steps, api_group_pool_items; schedule generation, queue, cooldown | | Collection | groups, posts | services/crawler.py, vk_parsers.py | communities, posts, post_media; source, rereading, normalization | | Profile and marketplace | groups tabs, market | community_profile.py, community_market.py, community_services.py | profile, products, services; filters and group association | | Quick selection | group-pool | fast_screen_pool.py, fast_wall_probe.py | fast_screen_pool_states, fast_screen_pool_checks; lease, due rows, retention | | Pipeline | pipeline | services/content_pipeline.py | content_pipeline_runs/posts/items; stages and persisted results | | OCR | processing, post details | ocr.py, ocr_jobs.py, cli/ocr_worker.py, cli/ocr_sidecar.py | ocr_jobs, post_media; completed/failed, attempts, timeout | | Canonical materials | results | publication_materials.py, publication-intelligence | publication_decisions/materials/media; decision-to-source relationship and revision | | Labeling and teacher | labels, processing | publication_teacher.py | teacher runs/results/applications; labeling, discrepancies, application | | Logs | logs, file-logs | web/app.py, helpers file logs | admin_job_logs, step logs, files; event and file traces | | Delivery | Telegram / MAX | separate bot repositories | canonical output + bot tables; downstream consumes prepared data | Before changing a feature, trace its interface, API, state and storage dependencies. Check inputs, errors, retries and related workflows. Update the map and affected diagrams alongside the code, rerun the related checks and attach evidence for the source version. ## Workflows and API 1. Browse anonymized schedules, groups, pipeline and materials. 2. Match a screen to its code-map entry and data diagram. 3. Open queue/history aggregates and read the measurement date and limitations. 4. Review the deletion record and disk/RAM comparison before and after decommissioning. ## Verification - 30 sanitized UI images checked before deletion - Deletion and authoritative DNS removal verified [CI: prod-deploy](https://github.com/komaroffsergei/wtg-s/actions/runs/27615782756) — `61c78116acaeb954dce480ac591249e4f5e5bf3e`, 2026-06-16T11:57:17Z; historical run of a different version. [CI: prod-deploy](https://github.com/komaroffsergei/wtg-s/actions/runs/27615134214) — `1299a357d0f54961a196d2eaecb714a1d5319b60`, 2026-06-16T11:45:01Z; historical run of a different version. ## Results and measurements - 30 meaningful screenshots of the actual interface retained. - Each screenshot includes its date, scenario and code revision. - 527,718 publications and 29,092 OCR jobs recorded; these are different entities, not quality metrics. - WTG was deleted after the local archive was verified; before cleanup, its directory occupied approximately 104 GiB. These checks apply to the listed version. Business metrics and model quality on an independent dataset were not measured. ![before after](https://portfolio.komaroff-dev.ru/evidence/wtg/charts/before-after.svg) [before after: source data](https://portfolio.komaroff-dev.ru/evidence/wtg/metrics/before-retirement.json) ![collection history](https://portfolio.komaroff-dev.ru/evidence/wtg/charts/collection-history.svg) [collection history: source data](https://portfolio.komaroff-dev.ru/evidence/wtg/metrics/before-retirement.json) ![container logs](https://portfolio.komaroff-dev.ru/evidence/wtg/charts/container-logs.svg) [container logs: source data](https://portfolio.komaroff-dev.ru/evidence/wtg/metrics/before-retirement.json) ![cpu samples](https://portfolio.komaroff-dev.ru/evidence/wtg/charts/cpu-samples.svg) [cpu samples: source data](https://portfolio.komaroff-dev.ru/evidence/wtg/metrics/before-retirement.json) ![material types](https://portfolio.komaroff-dev.ru/evidence/wtg/charts/material-types.svg) [material types: source data](https://portfolio.komaroff-dev.ru/evidence/wtg/metrics/before-retirement.json) ![ocr status](https://portfolio.komaroff-dev.ru/evidence/wtg/charts/ocr-status.svg) [ocr status: source data](https://portfolio.komaroff-dev.ru/evidence/wtg/metrics/before-retirement.json) ![table storage](https://portfolio.komaroff-dev.ru/evidence/wtg/charts/table-storage.svg) [table storage: source data](https://portfolio.komaroff-dev.ru/evidence/wtg/metrics/before-retirement.json) ## Limitations - Archived project; service decommissioned. There is no WTG demo link or recovery option. - Job statuses and confidence values are not measurements of OCR or model accuracy. - Legacy noVNC and unavailable sections are not presented as working features. - Observed web memory growth to approximately 3.8 GiB is documented as an observation; the code-level root cause was not established. ## Running and operating WTG is excluded from portfolio rollback. /opt/wtg, its containers, data, backups, access credentials, configurations and DNS records were deleted after the local archive was verified. GitHub source code and shared OS logs were retained as agreed. Dedicated deployment workflows were disabled and WTG keys removed. The result confirms logical deletion of managed data. Physical erasure of the provider's storage blocks cannot be confirmed from inside a guest VPS. The server checkout and published audited repository have different SHAs. Each evidence source is identified separately; no equivalence is claimed between the old image and the checkout contents. ## Screenshot index 1. [Collection and processing schedules](https://portfolio.komaroff-dev.ru/evidence/wtg/full/01-schedules.png) — 2026-09-06T21:16:48.230Z, source `21841487f52489d8bafaac379b9cc08199937661`. 2. [Account states; profiles and identifiers are hidden](https://portfolio.komaroff-dev.ru/evidence/wtg/full/02-accounts.png) — 2026-09-06T21:37:46.883Z, source `21841487f52489d8bafaac379b9cc08199937661`. 3. [Global group pool](https://portfolio.komaroff-dev.ru/evidence/wtg/full/03-group-pool.png) — 2026-09-06T21:16:57.865Z, source `21841487f52489d8bafaac379b9cc08199937661`. 4. [Processing, OCR queue and a retained teacher run](https://portfolio.komaroff-dev.ru/evidence/wtg/full/04-processing.png) — 2026-09-06T21:22:48.059Z, source `21841487f52489d8bafaac379b9cc08199937661`. 5. [Canonical publication materials](https://portfolio.komaroff-dev.ru/evidence/wtg/full/05-results.png) — 2026-09-06T21:37:50.167Z, source `21841487f52489d8bafaac379b9cc08199937661`. 6. [Community list](https://portfolio.komaroff-dev.ru/evidence/wtg/full/06-groups.png) — 2026-09-06T21:22:52.400Z, source `21841487f52489d8bafaac379b9cc08199937661`. 7. [Product catalog](https://portfolio.komaroff-dev.ru/evidence/wtg/full/07-market.png) — 2026-09-06T21:22:55.701Z, source `21841487f52489d8bafaac379b9cc08199937661`. 8. [Pipeline: operator overview](https://portfolio.komaroff-dev.ru/evidence/wtg/full/08-pipeline.png) — 2026-09-06T21:52:00.656620+00:00, source `21841487f52489d8bafaac379b9cc08199937661`. 9. [Labeling and tags](https://portfolio.komaroff-dev.ru/evidence/wtg/full/09-labels.png) — 2026-09-06T21:52:00.656620+00:00, source `21841487f52489d8bafaac379b9cc08199937661`. 10. [Job log](https://portfolio.komaroff-dev.ru/evidence/wtg/full/10-logs.png) — 2026-09-06T21:22:57.278Z, source `21841487f52489d8bafaac379b9cc08199937661`. 11. [File logs with sensitive sections redacted](https://portfolio.komaroff-dev.ru/evidence/wtg/full/11-file-logs.png) — 2026-09-06T21:23:03.321Z, source `21841487f52489d8bafaac379b9cc08199937661`. 12. [Community record: overview](https://portfolio.komaroff-dev.ru/evidence/wtg/full/12-group-overview.png) — 2026-09-06T21:23:06.094Z, source `21841487f52489d8bafaac379b9cc08199937661`. 13. [Community record: source publications](https://portfolio.komaroff-dev.ru/evidence/wtg/full/13-group-posts.png) — 2026-09-06T21:23:08.507Z, source `21841487f52489d8bafaac379b9cc08199937661`. 14. [Community record: products](https://portfolio.komaroff-dev.ru/evidence/wtg/full/14-group-market.png) — 2026-09-06T21:23:10.026Z, source `21841487f52489d8bafaac379b9cc08199937661`. 15. [Community record: profile](https://portfolio.komaroff-dev.ru/evidence/wtg/full/15-group-profile.png) — 2026-09-06T21:23:11.568Z, source `21841487f52489d8bafaac379b9cc08199937661`. 16. [Source publication and processing results](https://portfolio.komaroff-dev.ru/evidence/wtg/full/16-source-post.png) — 2026-09-06T21:23:13.647Z, source `21841487f52489d8bafaac379b9cc08199937661`. 17. [Pipeline item details](https://portfolio.komaroff-dev.ru/evidence/wtg/full/17-pipeline-item.png) — 2026-09-06T21:23:15.568Z, source `21841487f52489d8bafaac379b9cc08199937661`. 18. [Collection history and statistics](https://portfolio.komaroff-dev.ru/evidence/wtg/full/22-collection-stats.png) — 2026-09-06T21:23:19.083Z, source `21841487f52489d8bafaac379b9cc08199937661`. 19. [Disabled legacy VNC: historical states](https://portfolio.komaroff-dev.ru/evidence/wtg/full/23-vnc-status.png) — 2026-09-06T21:23:21.713Z, source `21841487f52489d8bafaac379b9cc08199937661`. 20. [Queues and operation aggregates after data loading](https://portfolio.komaroff-dev.ru/evidence/wtg/full/24-operations.png) — 2026-09-06T21:50:41.187Z, source `21841487f52489d8bafaac379b9cc08199937661`. 21. [Retained results of a local publication check](https://portfolio.komaroff-dev.ru/evidence/wtg/full/25-local-check.png) — 2026-09-06T21:23:25.630Z, source `21841487f52489d8bafaac379b9cc08199937661`. 22. [Groups in results](https://portfolio.komaroff-dev.ru/evidence/wtg/full/26-results-groups.png) — 2026-09-06T21:37:56.997Z, source `21841487f52489d8bafaac379b9cc08199937661`. 23. [Schedule history for 24 June 2026](https://portfolio.komaroff-dev.ru/evidence/wtg/full/27-schedule-history.png) — 2026-09-06T21:23:38.557Z, source `21841487f52489d8bafaac379b9cc08199937661`. 24. [Extracted products](https://portfolio.komaroff-dev.ru/evidence/wtg/full/28-results-products.png) — 2026-09-06T21:37:58.150Z, source `21841487f52489d8bafaac379b9cc08199937661`. 25. [Extracted services](https://portfolio.komaroff-dev.ru/evidence/wtg/full/29-results-services.png) — 2026-09-06T21:37:59.986Z, source `21841487f52489d8bafaac379b9cc08199937661`. 26. [Daily schedule charts](https://portfolio.komaroff-dev.ru/evidence/wtg/full/30-schedule-charts.png) — 2026-09-06T21:26:20.496Z, source `21841487f52489d8bafaac379b9cc08199937661`. 27. [Mobile results; documented table limitations](https://portfolio.komaroff-dev.ru/evidence/wtg/full/31-results-mobile.png) — 2026-09-06T21:38:03.366Z, source `21841487f52489d8bafaac379b9cc08199937661`. 28. [Telegram WebApp: city selection without a user session](https://portfolio.komaroff-dev.ru/evidence/wtg/full/32-telegram-profile.png) — 2026-09-06T21:50:51.803Z, source `21841487f52489d8bafaac379b9cc08199937661`. 29. [MAX WebApp: city selection without a user session](https://portfolio.komaroff-dev.ru/evidence/wtg/full/33-max-profile.png) — 2026-09-06T21:50:58.987Z, source `21841487f52489d8bafaac379b9cc08199937661`. 30. [Public “Gde Dvizh” landing page](https://portfolio.komaroff-dev.ru/evidence/wtg/full/34-gde-dviz-landing.png) — 2026-09-06T21:38:05.429Z, source `21841487f52489d8bafaac379b9cc08199937661`. Screenshots, diagrams and the original evidence package retain their source language. Captions and this document are translated into English. [Original evidence package](https://portfolio.komaroff-dev.ru/downloads/wtg-evidence.zip)