Skip to content

amlitio/construct-nomoly

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

FL Construction Anomaly Map (MVP)

A Florida-first interactive anomaly map (W.C. Dispatch–style) for construction contractors, permits, and entity networks.

What you get (ready to deploy)

  • Postgres + PostGIS schema + indexes + scoring snapshots
  • FastAPI backend with the requested endpoints:
    • /search?q=...
    • /license/{license_number}
    • /entity/{sunbiz_doc_number}
    • /address/{address_id}
    • /permit/{source}/{permit_id}
    • /map/permits?source=mdc|miami&bbox=minLon,minLat,maxLon,maxLat&...
    • /map/hotspots?metric=permit_volume|entity_density&time_window=90d
    • /score/license/{license_id}, /score/address/{address_id}
  • ETL scripts:
    • sunbiz_loader.py (load from Sunbiz quarterly downloads you provide or direct URL)
    • arcgis_mdc_loader.py (ArcGIS REST pull)
    • arcgis_miami_loader.py (ArcGIS REST pull)
    • dbpr_on_demand.py (on-demand cached DBPR fetch; minimal MVP parser)
  • Next.js frontend (Mapbox GL) with:
    • Map landing page + filters
    • Detail pages for license/entity/address/permit
    • Score panel + explainable reasons

⚠️ Legal note: this app surfaces investigative indicators (triage signals), not proof of wrongdoing. The UI includes disclaimers by design.


Quickstart (local)

1) Requirements

  • Docker + Docker Compose
  • A Mapbox token (for the map UI)

2) Configure env

Copy .env.example to .env and set:

  • NEXT_PUBLIC_MAPBOX_TOKEN

3) Run

docker compose up --build

Load data (ETL)

Open a shell in the API container:

docker compose exec api bash

Sunbiz (quarterly download)

  1. Download Sunbiz datasets from:
  1. Put files into backend/data/sunbiz/ (mounted inside the container if you volume mount, or copy into container)
  2. Run:
python -m app.etl.sunbiz_loader --entities backend/data/sunbiz/<ENTITIES_FILE> --officers backend/data/sunbiz/<OFFICERS_FILE>

Miami-Dade permits (ArcGIS)

python -m app.etl.arcgis_mdc_loader

City of Miami permits (ArcGIS)

python -m app.etl.arcgis_miami_loader

DBPR is fetched on-demand when the user searches/licenses are requested, and cached in DB.


Deployment (production)

  • Build and deploy with any Docker-capable platform (Render, Fly.io, ECS, GCP, etc.)
  • Put Postgres on a managed service if preferred
  • Set env vars:
    • DATABASE_URL (point to your Postgres)
    • NEXT_PUBLIC_API_BASE (public API URL)
    • NEXT_PUBLIC_MAPBOX_TOKEN

Disclaimers / UI copy

  • “Signals are investigative indicators, not proof.”
  • “Scores are explainable and reproducible; click through to supporting data.”

Automation / “AI agents” (what is automated now)

This repo now includes a task queue (Celery + Redis) and scheduled jobs (Celery Beat) so the platform can run unattended:

Background services

  • redis: task broker
  • worker: executes jobs
  • beat: schedules recurring jobs

Scheduled jobs (defaults)

  • Weekly refresh: Miami-Dade permits (Sun 3:00 UTC)
  • Weekly refresh: City of Miami permits (Sun 4:00 UTC)
  • Nightly linking: license↔entity and permit↔license
  • Nightly score recompute: latest licenses/addresses

Manual triggers (admin endpoints)

All require header: X-API-Key: $ADMIN_API_KEY

  • POST /admin/tasks/mdc/refresh
  • POST /admin/tasks/miami/refresh
  • POST /admin/tasks/linking/run
  • POST /admin/tasks/scores/recompute

Optional “AI agents”

LLM agents are optional and used only for messy parsing / entity-resolution fallbacks. Enable by setting:

  • LLM_PROVIDER=openai
  • OPENAI_API_KEY=...
  • OPENAI_MODEL=...

Current agent scaffolding:

  • app/agents/dbpr_parser_agent.py: extract structured fields from DBPR HTML when deterministic parsing fails.

For production, you should still implement deterministic extraction first, then use the agent only as a fallback.

Ops Console

Open the Ops Console at http://localhost:3000/ops.

  • Paste ADMIN_API_KEY into the Ops Console field.
  • Trigger refresh/linking/scoring jobs.
  • Review pending matches and approve/reject them.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors