mxpic_EDA/develop_plan/project_understanding.md

# mxpic_EDA Project Understanding

## Purpose

`mxpic_EDA` is a local/intranet EDA web application for creating photonic integrated circuit layout projects. It combines:

- A Flask backend for authentication, project APIs, PDK browsing, layout persistence, preview generation, and GDS export.
- Static frontend pages for login, dashboard, and canvas editing.
- SQLite account/audit storage plus filesystem-based project layout storage.
- External build-time tooling (`mxpic_router`, Nazca, and sometimes `gdstk`) for routed SVG preview and final GDS generation.

The project is designed so normal login, dashboard, canvas editing, YAML generation, and PDK browsing can run without importing the router stack. Build actions load those heavier dependencies only when needed.

## Main Entry Points

### Start the App

The intended intranet startup path is:

```powershell
.\run_intranet_server.ps1
```

That script sets:

- `MXPIC_HOST=0.0.0.0`
- `MXPIC_PORT=3000`
- `MXPIC_DEBUG=0`
- a fallback `MXPIC_SECRET_KEY` if none is set

Then it runs:

```powershell
python backend\server.py
```

The server listens on:

```text
http://<host-computer-ip>:3000
```

Default local accounts are:

```text
admin / 123456
engineer / 123456
```

`admin` is a manager account. `engineer` is a developer account.

## Repository Structure

```text
backend/
  server.py                 Flask app, routes, project/layout APIs
  database.py               SQLite initialization, users, profiles, audit logs
  pdk_access.py             Role-aware PDK root selection and export helpers
  gds_builder.py            Final GDS wrapper around mxpic_router
  routed_layout_preview.py  SVG preview generation via temporary routed GDS
  router_dependency.py      Lazy build-time dependency validation
  technology_manifest.py    technology.yml loading/validation
  icons/                    Category icons served by backend

frontend/
  login.html                Login page
  dashboard.html            Project/account dashboard
  canvas.html               Main layout editor
  canvas-helpers.js         Canvas/YAML/route helper functions and tests target

database/
  mxpic_data.db             SQLite app database
  <username>/layout/...     Saved project metadata, YAML cells, SVG previews
  _exports/<uuid>/...       Temporary generated GDS downloads

docs/
  INTRANET_DEPLOYMENT.md
  GDS_SVG_GENERATION_LOGIC.md
  PDK_TECHNOLOGY_XSECTION_LOADING.md

tests/
  JavaScript assertion tests for static wiring and canvas helpers
```

## User Workflow

1. User opens `/`.
2. If not logged in, Flask serves `frontend/login.html`.
3. Login form posts credentials to `POST /login`.
4. On success, the session stores user id, username, and user group.
5. User lands on `/dashboard`.
6. Dashboard loads:
   - `/api/profile`
   - `/api/logs`
   - `/api/projects`
   - `/api/technologies`
7. User creates or opens a project.
8. Creating a project posts `{ name, technology }` to `POST /api/projects`.
9. Opening a project navigates to `/canvas?project=<project>`.
10. Canvas loads project data from `/api/projects/<project>`.
11. Canvas loads the selected `technology.yml` manifest from `/api/technologies/<foundry>/<technology>/manifest`.
12. Canvas loads the project-scoped PDK library from `/api/library?project=<project>`.
13. User places components, elements, pins, and routes.
14. Save or Build Layout writes YAML to disk through `/api/save-layout`.
15. Build Layout also generates and opens an SVG preview.
16. Build GDS calls `/api/build-gds` and downloads the generated GDS.

## Project and Data Model

Each user's projects live under:

```text
database/<username>/layout/<project>/
```

Project metadata is stored in:

```text
database/<username>/layout/<project>/.project.json
```

Example:

```json
{
  "name": "mxpic_project_1",
  "technology": "Silterra/EMO1_2ML_CU_Al_RDL"
}
```

Saved cells are YAML files:

```text
database/<username>/layout/<project>/<cell>.yml
```

Generated previews are SVG files:

```text
database/<username>/layout/<project>/<cell>.svg
```

Modern saved layout YAML contains:

- `schema_version`
- `kind`
- `coordinate_system`
- `canvas_size`
- `project`
- `name`
- `type`
- `version`
- `pins`
- `instances`
- `elements`
- `bundles`

`instances` reference basic components, forge/generated components, other project cells, or PDK component paths. `bundles` store routed links and route settings such as `xsection`, `family`, `width`, `radius`, and `routing_type`.

## PDK and Technology Workflow

The active PDK root is chosen by user role:

- `manager`: `MXPIC_PDK_ATLAS_ROOT` or `opt_pdk_atlas/foundries`
- `developers` and `user`: `MXPIC_PDK_PUBLIC_ROOT` or `opt_pdk_public/foundries`

The dashboard lists available technologies by scanning:

```text
<active_role_pdk_root>/<foundry>/<technology>/technology.yml
```

The selected technology is saved in `.project.json`.

Canvas then uses that selected technology to:

- Load route defaults and xsections from `technology.yml`.
- Scope the component library browser to the chosen foundry/technology.
- Save component paths that can later resolve under the base role PDK root.

At build time, the router receives the active PDK root and the selected project's technology manifest path. The router applies technology layers/xsections to Nazca before resolving PDK assets and routing links.

## Build Layout vs Save vs Build GDS

### Save

The canvas "Save" path calls `handleSaveProjectLayouts`. It writes YAML for every editable project/cell page to `/api/save-layout` with preview disabled. It is for persistence, not preview.

### Build Layout

The canvas "Build Layout" path:

1. Validates route crossings.
2. Serializes the active page to YAML.
3. Posts to `/api/save-layout`.
4. Backend writes `<cell>.yml`.
5. Backend writes route sidecar data when available.
6. Backend calls `routed_layout_preview.py`.
7. `routed_layout_preview.py` calls `mxpic_router.build_project_gds`.
8. The temporary routed GDS is converted to `<cell>.svg` with `gdstk`.
9. Frontend opens the SVG preview tab.

### Build GDS

The canvas "Build GDS" path:

1. Validates route crossings across editable pages.
2. Posts `{ project }` to `/api/build-gds`.
3. Backend reads saved YAML files already on disk.
4. Backend creates `database/_exports/<uuid>/<project>.gds`.
5. `backend/gds_builder.py` delegates to `mxpic_router.build_project_gds`.
6. Backend returns `download_url`.
7. Frontend triggers a browser download.
8. Backend removes the temporary export folder after serving the file.

Important: Build GDS does not serialize unsaved in-memory canvas state first. Users should Save or Build Layout before Build GDS if they want latest canvas edits included.

## Backend API Map

Primary pages:

- `GET /`
- `POST /login`
- `GET /dashboard`
- `GET /canvas`
- `GET /canvas-helpers.js`
- `GET /logout`

Account and activity:

- `GET /api/health`
- `GET/PATCH /api/profile`
- `POST /api/profile/password`
- `GET/POST /api/logs`

Projects and cells:

- `GET /api/projects`
- `POST /api/projects`
- `GET /api/projects/<project_name>`
- `DELETE /api/projects/<project_name>`
- `PATCH/DELETE /api/projects/<project_name>/cells/<cell_name>`
- `POST /api/save-layout`
- `GET /api/projects/<project_name>/cells/<cell_name>/layout.svg`

Build/export:

- `POST /api/build-gds`
- `GET /api/exports/<export_id>/<filename>`
- `GET /api/projects/<project_name>/gds/<filename>`

PDK/technology/library:

- `GET /api/technologies`
- `GET /api/technologies/<foundry>/<technology>/manifest`
- `GET /api/library`
- `GET /api/component/<component_name>`
- `GET /api/component/<component_name>/image`
- `GET /api/icon/<category>`

## Persistence and Accounts

SQLite database:

```text
database/mxpic_data.db
```

Tables observed:

- `users`
- `user_logs`

The backend initializes the database on startup. It also performs lightweight migrations for profile, credit, occupation, and group fields.

User groups affect PDK access:

- `manager` can use atlas/private PDK roots and prefers full GDS assets.
- `developers` and `user` use the public PDK roots.

Audit logs record user actions with project/cell context, details, IP address, and UTC timestamp.

## Tests

Tests are JavaScript assertion scripts under `tests/`.

The suite mostly checks integration contracts by reading source files, including:

- API route strings and handler wiring.
- Build Layout and Build GDS frontend/backend wiring.
- Removal of legacy local preview/PDK registry modules.
- Router-backed GDS and SVG preview paths.
- Technology manifest and PDK access behavior.
- Canvas helper behavior such as pin/handle placement, YAML serialization, route defaults, route YAML, element ports, and route crossing checks.

Likely command:

```powershell
node --test tests
```

## Current Repository State Notes

This checkout contains unresolved merge-conflict markers in some tracked files:

- `database/engineer/layout/mxpic_project_1/mxpic_project_1.yml`
- `database/engineer/layout/mxpic_project_1/mxpic_project_1.svg`
- `tests/layout-ui-wiring.test.js`
- `tests/layout-backend-static.test.js`

That means some checked sample data may not parse as YAML/SVG, and the test suite may fail before reaching assertions until those files are cleaned.

The checked admin and engineer project metadata both select:

```text
Silterra/EMO1_2ML_CU_Al_RDL
```

## Mental Model

Think of this app as:

```text
Flask session/account layer
  -> role-scoped PDK/technology selection
  -> dashboard project metadata
  -> canvas layout editing
  -> YAML saved on disk
  -> mxpic_router/Nazca build path
  -> SVG preview or downloadable GDS
```

The repository itself owns the web UI, persistence, API contracts, and orchestration. The actual routed photonic layout build is intentionally delegated to the external `mxpic_router` stack.