# TenderLens
Technical documentation
HTML is split into chunks and indexed through a queue. Search runs in pgvector; answers link to the retrieved chunks. The demo has four test documents. Embeddings and the LLM are mocked; the external crawler is disabled.
Source version: `a618d1e7be8ac2d9ffc7b6818dc5c6b550089e16`
Implementation history: since 20 August 2026. Public demo: 7 September 2026.
[Project notes](https://portfolio.komaroff-dev.ru/en/projects/tenders/) · [Online documentation](https://portfolio.komaroff-dev.ru/en/projects/tenders/docs/)
[Demo](https://tenders.komaroff-dev.ru/)
[Source repository](https://github.com/komaroffsergei/tender-lens)
## Task
Search over a fixed document set. Answers need links to the source chunks. If there are no relevant matches, the response says so.
Searching and reading a document collection.
## Work done
- Document normalization, event-based indexing and chunk storage.
- Search and answer API with source links. A separate response for empty context.
- Test corpus, deterministic AI mocks and per-session limits.
## Stack
| Technology | Role |
| --- | --- |
| FastAPI / Pydantic | HTTP contracts, validation and search responses. |
| NATS JetStream | Event delivery to the indexer and replay. |
| PostgreSQL / pgvector | Documents, attachments, chunks and cosine search. |
| SQLAlchemy / Python | Normalization, extraction, chunking and content_hash verification. |
## Architecture and decisions
- Seed sends a real TenderChangedV1 event through NATS; the indexer extracts HTML and creates chunks.
- PostgreSQL VECTOR(1024) performs cosine search. FakeAIProvider creates embeddings using the hashing trick.
- Event replay uses content_hash: an unchanged document does not duplicate chunks.
- Answers use an explicitly labeled template based on retrieved passages; the external crawler and Ollama are disabled.
### Components of the published demo
[Mermaid source](https://portfolio.komaroff-dev.ru/evidence/tenders/diagrams/components.mmd)
### Main workflow
[Mermaid source](https://portfolio.komaroff-dev.ru/evidence/tenders/diagrams/scenario.mmd)
### Deployment of the published version
[Mermaid source](https://portfolio.komaroff-dev.ru/evidence/tenders/diagrams/deployment.mmd)
### Data structure and storage boundaries
[Mermaid source](https://portfolio.komaroff-dev.ru/evidence/tenders/diagrams/data.mmd)
### Original project integrations. Disabled in the demo.
[Mermaid source](https://portfolio.komaroff-dev.ru/evidence/tenders/diagrams/original.mmd)
## Code map
| Feature | Screen / API | Module | Data / verification |
| --- | --- | --- | --- |
| Initial indexing | CLI portfolio | cli._upsert_demo_record, NatsBroker, IndexerService | sources, tenders, attachments, chunks |
| Search / answer | api/v1/search, ask | SearchService, FakeAIProvider | chunks VECTOR(1024) |
| Sources | local HTML | web/demo | original corpus |
| Event replay | api/demo/replay | NatsBroker / hash guard | shared read-only corpus |
| Session and rate limit | cookie / middleware | portfolio, auth, rate_limit | api_keys (hash and counters only) |
Before changing a feature, trace its interface, API, state and storage dependencies. Check inputs, errors, retries and related workflows. Update the map and affected diagrams alongside the code, rerun the related checks and attach evidence for the source version.
## Workflows and API
1. Find a document about servers or an Angular portal.
2. Ask a question and open the answer's sources.
3. Enter a query outside the corpus and check the missing-context response.
4. Replay indexing events: the four documents and 16 chunks should remain without duplication.
## Verification
- 118 unit tests passed; 18 integration/external checks were excluded from this run.
- Search and Ask return sources; unrelated query returns no context
- Queue replay preserves 16 chunks and ready status
- Independent signed expiring HttpOnly sessions
- No verified GitHub CI run is available for this SHA; the listed evidence covers local checks and published-version scenarios. Historical CI links, where available, are labeled separately.
[CI: CI](https://github.com/komaroffsergei/tender-lens/actions/runs/33046876189) — `84c37f0f6b52fef9ec0eddb11078ebed0b5aef26`, 2026-08-27T06:43:00Z; historical run of a different version.
[CI: Documentation](https://github.com/komaroffsergei/tender-lens/actions/runs/33046876185) — `84c37f0f6b52fef9ec0eddb11078ebed0b5aef26`, 2026-08-27T06:43:00Z; historical run of a different version.
## Results and measurements
- 12 meaningful screenshots of the actual interface retained.
- Each screenshot includes its date, scenario and code revision.
- Peak container RAM in the short test: 127.04 MiB. Shared services are counted separately; the shop includes its own database.
- 6 scenario repetitions measured; median 997.0 ms, maximum 2738 ms. Timing includes the browser and HTTPS; methodology and limitations are included.
- Four ready documents, 16 chunks. Replaying four NATS events created no duplicates.
These checks apply to the listed version. Business metrics and model quality on an independent dataset were not measured.
[Total project container resources · 2026-09-07T01:08:27 — 01:11:22 UTC: source data](https://portfolio.komaroff-dev.ru/evidence/tenders/metrics/measurements.json)
[Measured scenario repetitions over HTTPS · 2026-09-07T01:08:36 UTC; methodology limitations in JSON: source data](https://portfolio.komaroff-dev.ru/evidence/tenders/metrics/measurements.json)
## Limitations
- Four synthetic documents and 16 chunks form a demonstration corpus, not a database of real procurement records.
- Cosine scores and template answers are not an evaluation of a production model's quality.
- Up to 30 requests per minute per session and 1,000 active demo keys. User queries are not stored.
- Worker health confirms the connection at startup; a separate stalled-computation detector is not implemented.
## Running and operating
The original project has a crawler and an AI provider integration. The demo disables live AI in its config and only accepts documents from the test corpus.
Dockerfile.portfolio is built outside the VPS; the API and worker share one pinned image with separate limits. PostgreSQL and NATS are accessible only on the internal network; the API is served through localhost:18403 and Nginx.
Run migrations and the portfolio seed CLI at startup. API readiness checks its dependencies. Reindexing is safe because of content_hash; after a restart, verify four ready documents and 16 chunks. Session ApiKeys expire by TTL and are cleaned up every five minutes.
Builds run outside the VPS; the server receives static files and versioned images. Secrets are kept separately. Logs rotate; changing log options requires recreating the container. No server-wide docker prune.
The code map links a screen or API to its module and data. A change needs those links checked, the map updated and the affected flows rerun: normal path, error, retry and reset. Diagrams can go stale, so their version is listed alongside them.
Rollback checked at 2026-09-07T01:04:24.534304+00:00: 20260907-verified → 20260907c → 20260907-verified. HTTPS, readiness and SHA were confirmed after each switch. This checks configuration and artifacts at one verified SHA; no reverse database migration was performed.
## Screenshot index
1. [Search and explicit demo boundaries](https://portfolio.komaroff-dev.ru/evidence/tenders/full/01-overview.png) — 2026-09-06T23:42:53.262Z, source `a618d1e7be8ac2d9ffc7b6818dc5c6b550089e16`.
2. [Actual states of four documents and 16 chunks in PostgreSQL](https://portfolio.komaroff-dev.ru/evidence/tenders/full/02-index-status.png) — 2026-09-06T23:42:53.631Z, source `a618d1e7be8ac2d9ffc7b6818dc5c6b550089e16`.
3. [Cosine search result. Embeddings are mocked.](https://portfolio.komaroff-dev.ru/evidence/tenders/full/03-server-search.png) — 2026-09-06T23:42:54.156Z, source `a618d1e7be8ac2d9ffc7b6818dc5c6b550089e16`.
4. [Passage, attachment name, cosine score and source link](https://portfolio.komaroff-dev.ru/evidence/tenders/full/04-result-detail.png) — 2026-09-06T23:42:54.242Z, source `a618d1e7be8ac2d9ffc7b6818dc5c6b550089e16`.
5. [Original synthetic HTML source](https://portfolio.komaroff-dev.ru/evidence/tenders/full/05-source.png) — 2026-09-06T23:42:54.468Z, source `a618d1e7be8ac2d9ffc7b6818dc5c6b550089e16`.
6. [Template answer based on retrieved sources](https://portfolio.komaroff-dev.ru/evidence/tenders/full/06-answer.png) — 2026-09-06T23:42:54.982Z, source `a618d1e7be8ac2d9ffc7b6818dc5c6b550089e16`.
7. [Another scenario: Angular portal requirements](https://portfolio.komaroff-dev.ru/evidence/tenders/full/07-angular-answer.png) — 2026-09-06T23:42:55.514Z, source `a618d1e7be8ac2d9ffc7b6818dc5c6b550089e16`.
8. [No answer when suitable passages are missing](https://portfolio.komaroff-dev.ru/evidence/tenders/full/08-no-context.png) — 2026-09-06T23:42:55.991Z, source `a618d1e7be8ac2d9ffc7b6818dc5c6b550089e16`.
9. [Minimum query length validation](https://portfolio.komaroff-dev.ru/evidence/tenders/full/09-validation.png) — 2026-09-06T23:42:56.427Z, source `a618d1e7be8ac2d9ffc7b6818dc5c6b550089e16`.
10. [Real replay of four events through NATS with an idempotent worker](https://portfolio.komaroff-dev.ru/evidence/tenders/full/10-replay.png) — 2026-09-06T23:42:57.274Z, source `a618d1e7be8ac2d9ffc7b6818dc5c6b550089e16`.
11. [Mobile interface, mock disclosures and query form](https://portfolio.komaroff-dev.ru/evidence/tenders/full/11-mobile-form.png) — 2026-09-06T23:42:59.048Z, source `a618d1e7be8ac2d9ffc7b6818dc5c6b550089e16`.
12. [Mobile result cards](https://portfolio.komaroff-dev.ru/evidence/tenders/full/12-mobile-results.png) — 2026-09-06T23:42:59.587Z, source `a618d1e7be8ac2d9ffc7b6818dc5c6b550089e16`.
Screenshots, diagrams and the original evidence package retain their source language. Captions and this document are translated into English.
[Original evidence package](https://portfolio.komaroff-dev.ru/downloads/tenders-evidence.zip)