mxpic_EDA/PDK_TECHNOLOGY_XSECTION_LOADING.md

# PDK, Technology, and Xsection Loading Logic Path

This document explains how the current code loads:

- PDK component information: component folders, component YAML, images, and GDS assets.
- Technology information: selected foundry/technology for each project.
- Xsection information: routing cross-section defaults and Nazca layer bindings from `technology.yml`.

Line numbers refer to the current codebase at the time this document was written.

## Mental Model

There are three related but separate data paths.

1. Component PDK assets
   - Source: role-controlled PDK trees:
     - `opt_pdk_public/foundries`
     - `opt_pdk_atlas/foundries`
   - Controlled by user group through `pdk_root_for_session`
     (`backend/pdk_access.py` line 43).
   - Used for library browsing, component metadata, component images, and final
     component GDS resolution.

2. Technology registry and manifests
   - Source: the same role-controlled PDK trees as component assets.
   - Path: `<active_role_pdk_root>/<foundry>/<technology>/technology.yml`.
   - Used to list available technologies, save the project technology choice,
     load route defaults, and register Nazca layers/xsections.

3. Saved project layout YAML
   - Source: `database/<username>/layout/<project>/<cell>.yml`.
   - Contains placed component paths and `bundles.*.links[*].xsection`.
   - Consumed by `mxpic_router` when building SVG preview GDS or downloadable GDS.

Important distinction:

- The active role PDK root is now the single source of truth.
- Normal/developer users read from `opt_pdk_public/foundries`.
- Manager users read from `opt_pdk_atlas/foundries`.
- The selected project technology scopes the library browser and selects which
  `technology.yml` is loaded from that same role root.

## Active Roots

Backend startup defines the important paths:

- `REPO_ROOT` (`backend/server.py` line 36) points two levels above
  `backend/`.
- There is no separate technology-manifest copy inside `mxpic_EDA`.

Role-based PDK root:

`Flask session user_group` -> `current_pdk_root`
(`backend/server.py` line 196) -> `pdk_root_for_session`
(`backend/pdk_access.py` line 43) -> `pdk_root_for_group`
(`backend/pdk_access.py` line 27) : choose:

- manager -> `MXPIC_PDK_ATLAS_ROOT` or `../opt_pdk_atlas/foundries`.
- developers/user -> `MXPIC_PDK_PUBLIC_ROOT` or `../opt_pdk_public/foundries`.

## Technology Selection Path

Compact path:

`Dashboard opens` -> `loadTechnologies` (`frontend/dashboard.html` line 1238)
-> GET `/api/technologies` -> `list_technologies`
(`backend/server.py` line 441) : scan `current_pdk_root()` ->
return `{ foundry, technology, id, label }` -> dashboard fills the technology
select.

Detailed path:

1. The dashboard calls `/api/technologies`:
   - `loadTechnologies` starts at `frontend/dashboard.html` line 1238.
   - It stores `data.technologies` and creates `<option>` values using
     `tech.id`, for example `Silterra/EMO1_2ML_CU_Al_RDL`.

2. The backend scans the technology registry:
   - Route: `/api/technologies` (`backend/server.py` line 439).
   - Function: `list_technologies` (`backend/server.py` line 441).
   - It scans `current_pdk_root()`.
   - A technology directory is exposed only when it contains `technology.yml`.
   - For each valid directory pair it returns:
     - `foundry`
     - `technology`
     - `id: foundry/technology`
     - `label: foundry / technology`

3. Creating a project stores the selected technology:
   - Dashboard create click posts to `/api/projects`
     (`frontend/dashboard.html` line 1303).
   - Body contains `{ name, technology }`.
   - Backend route: `/api/projects` (`backend/server.py` line 603).
   - Function: `create_project` (`backend/server.py` line 605).
   - It writes `.project.json` with:
     - `name`
     - `technology`
   - Metadata helper path: `project_meta_path`
     (`backend/server.py` line 228).
   - Metadata writer: `write_project_meta`
     (`backend/server.py` line 242).

Saved metadata location:

`database/<username>/layout/<project>/.project.json`

Example:

```json
{
  "name": "mxpic_project_1",
  "technology": "Silterra/EMO1_2ML_CU_Al_RDL"
}
```

## Technology Manifest Loading Path

Compact path:

`Canvas opens project` -> `loadProject` (`frontend/canvas.html` line 5030)
-> GET `/api/projects/<project>` -> `get_project`
(`backend/server.py` line 632) : returns saved `technology` -> frontend
`loadTechnologyManifest` (`frontend/canvas.html` line 4056) -> GET
`/api/technologies/<foundry>/<technology>/manifest` -> `read_technology_manifest`
(`backend/technology_manifest.py` line 27) -> return full `technology.yml`
manifest.

Detailed path:

1. Canvas loads project metadata:
   - `loadProject` fetches `/api/projects/<project>`
     (`frontend/canvas.html` line 5033).
   - Backend `get_project` returns:
     - project name
     - saved cells
     - `technology: read_project_meta(project_name).get("technology")`
       (`backend/server.py` line 653).

2. Canvas loads the selected technology manifest:
   - `loadTechnologyManifest` starts at `frontend/canvas.html` line 4056.
   - It expects technology IDs in `foundry/technology` format.
   - It fetches:
     `/api/technologies/<foundry>/<technology>/manifest`.
   - If missing or invalid, it logs an error and uses
     `FALLBACK_TECHNOLOGY_MANIFEST`.

3. Backend validates and returns the manifest:
   - Route:
     `/api/technologies/<foundry>/<technology>/manifest`
     (`backend/server.py` line 465).
   - Function: `get_technology_manifest`
     (`backend/server.py` line 467).
   - It calls `read_technology_manifest`
     (`backend/technology_manifest.py` line 27).
   - The expected path is built by `technology_manifest_path`
     (`backend/technology_manifest.py` line 16):

```text
<active_role_pdk_root>/<foundry>/<technology>/technology.yml
```

4. Backend manifest validation:
   - `technology.yml` must exist.
   - `manifest.xsections` must be a dictionary.
   - `manifest.defaults` must be a dictionary.
   - Otherwise the API returns 404 with a `TechnologyManifestError`.

## Current `technology.yml` Structure

Active example:

`opt_pdk_public/foundries/Silterra/EMO1_2ML_CU_Al_RDL/technology.yml`

For manager sessions, the equivalent file is read from
`opt_pdk_atlas/foundries`.

Important sections:

- `schema_version` (`technology.yml` line 7).
- `foundry` (`technology.yml` line 8).
- `technology` (`technology.yml` line 9).
- `layers` (`technology.yml` line 18).
- `routing_types` (`technology.yml` line 41).
- `defaults` (`technology.yml` line 44).
- `xsections` (`technology.yml` line 50).

Current xsections:

- `strip` (`technology.yml` line 51).
- `rib_low` (`technology.yml` line 58).
- `metal_1` (`technology.yml` line 67).
- `metal_2` (`technology.yml` line 74).

Meaning of the main fields:

- `layers`: maps technology layer names to GDS layer/datatype pairs.
- `routing_types`: choices shown by the route editor.
- `defaults`: default route `xsection`, `family`, `width`, `radius`, and
  `routing_type`.
- `xsections`: per-cross-section defaults and layer bindings.
- `constants` and `source_class`: currently kept as manifest metadata. The
  active frontend and router code do not consume them directly.

## Frontend Xsection Usage

Compact path:

`technology.yml xsections` -> backend manifest API -> `technologyManifest`
state (`frontend/canvas.html` line 3756) -> route editor dropdown and route
defaults -> saved YAML `bundles.output_bus.links[*].xsection`.

Detailed path:

1. Fallback manifest:
   - `FALLBACK_TECHNOLOGY_MANIFEST` is defined in
     `frontend/canvas-helpers.js` line 105.
   - It includes fallback `strip`, `rib_low`, `metal_1`, and `metal_2`
     xsections.

2. Canvas state:
   - `projectTechnology` state is declared at `frontend/canvas.html` line 3755.
   - `technologyManifest` state is declared at `frontend/canvas.html` line 3756.

3. Route defaults:
   - `getXsectionInfo` (`frontend/canvas-helpers.js` line 132) looks up
     `manifest.xsections[xsection]`.
   - `createRouteSettings` (`frontend/canvas-helpers.js` line 138) merges:
     - manifest defaults
     - selected xsection defaults
     - user overrides saved on the edge
   - Resulting route state always has:
     - `xsection`
     - `family`
     - `width`
     - `radius`
     - `routing_type`

4. Changing xsection in the route editor:
   - Right panel builds available xsections from
     `Object.keys(technologyManifest.xsections)`
     (`frontend/canvas.html` line 2999).
   - The xsection select calls `updateRouteXsection`
     (`frontend/canvas.html` line 3037).
   - `updateRouteXsection` refreshes family, default width, and default radius
     (`frontend/canvas-helpers.js` line 167).

5. Link toolbar choices:
   - `linkXsectionChoices` is built from `technologyManifest.xsections`
     (`frontend/canvas.html` line 3775).
   - It prefers the order `strip`, `rib_low`, `metal_1`, `metal_2`, then any
     extra xsections.

## Component PDK Library Loading Path

Compact path:

`Canvas needs library` -> `fetchLibrary` (`frontend/canvas.html` line 4576)
-> GET `/api/library?project=<project>` -> `pdk_root_for_request_project`
(`backend/server.py` line 222) -> `scoped_pdk_root_for_project`
(`backend/server.py` line 201) -> `findComps`
(`backend/server.py` line 262) -> frontend library tree.

Detailed path:

1. Frontend requests the library:
   - `fetchLibrary` calls `/api/library?project=<currentProjectName>`
     (`frontend/canvas.html` line 4576).

2. Backend chooses the PDK root for this request:
   - Route: `/api/library` (`backend/server.py` line 854).
   - Function: `getLib` uses `pdk_root_for_request_project`.
   - `pdk_root_for_request_project` starts at `backend/server.py` line 222.
   - If a project is supplied, it calls `scoped_pdk_root_for_project`.
   - Otherwise it uses `current_pdk_root`.

3. Project scoping:
   - `scoped_pdk_root_for_project` starts at `backend/server.py` line 201.
   - It reads the project `.project.json` technology field.
   - It splits the value into `foundry/technology`.
   - It scopes the role root to:

```text
<active_role_pdk_root>/<foundry>/<technology>
```

   - It only returns that scoped path if it exists and stays under the active
     role root.
   - If not valid, it falls back to the active role root.

4. Component scanning:
   - `findComps` starts at `backend/server.py` line 262.
   - It walks the chosen PDK directory.
   - Any folder containing a `.yml` file is treated as one component leaf.
   - It stores component metadata:
     - folder path
     - first component YAML filename
     - category
     - relative component path
   - `addCompsToTree` (`backend/server.py` line 290) converts this flat map
     into the nested library tree returned to the canvas.

Important detail:

`getLib` scans the project-scoped directory but calls:

```python
findComps(comps_root, current_pdk_root())
```

This means the visible library is limited to the selected project technology,
but each component `__path__` is stored relative to the base role PDK root, not
relative to the scoped technology directory.

For example, if the active role PDK root is:

```text
opt_pdk_public/foundries
```

and the project technology is:

```text
Silterra/EMO1_2ML_CU_Al_RDL
```

a component path saved into layout YAML can include:

```text
Silterra/EMO1_2ML_CU_Al_RDL/<category>/<component>/<component>
```

This is important because build and preview pass the base role PDK root into
`mxpic_router`, and the saved component path can still resolve under that base
root.

## Component Metadata and Image Loading

Component YAML path:

`Canvas load/drop component` -> `loadComponentMetadata`
(`frontend/canvas.html` line 4082) -> GET
`/api/component/<component_name>?project=<project>` -> `readCompYaml`
(`backend/server.py` line 345) -> return component YAML.

Details:

- Route: `/api/component/<component_name>` (`backend/server.py` line 869).
- The backend uses the same project-scoped PDK root logic as `/api/library`.
- `readCompYaml` walks the selected PDK root and matches the component folder
  basename.
- If the PDK root is under `opt_pdk_public` or `opt_pdk_atlas`, legacy
  component metadata with `ports` is normalized to `pins`.

Component image path:

`Canvas thumbnail` -> `/api/component/<component_name>/image?project=<project>`
-> `find_component_dir` (`backend/server.py` line 367) -> first
`.png`, `.jpg`, `.jpeg`, or `.svg` in the component folder.

Route:

- `/api/component/<component_name>/image` (`backend/server.py` line 878).

## Saved YAML: How PDK Paths and Xsections Are Written

Compact path:

`Build Layout` or `Save` -> `buildYamlForPage`
(`frontend/canvas.html` line 6141) -> `findComponentPath`
(`frontend/canvas.html` line 3659) -> `buildInstancesYaml`
(`frontend/canvas-helpers.js` line 665) -> saved `instances.*.component`.

Component path details:

- `findComponentPath` searches the loaded library tree.
- If a component node has `__path__`, that path is used.
- `buildInstanceYaml` (`frontend/canvas-helpers.js` line 643) writes:

```yaml
instances:
  <instance_name>:
    component: <component path from library>
```

- Forge and basic components are special:
  - Forge components save the forge component type.
  - Basic components save names such as `waveguide`, `90 bend`, or `taper`.
  - Normal PDK components save the PDK library path.

Xsection path details:

`Canvas edges` -> `buildBundlesYaml` (`frontend/canvas-helpers.js` line 988)
-> `createRouteSettings` (`frontend/canvas-helpers.js` line 138) -> saved
bundle links:

```yaml
bundles:
  output_bus:
    routing_type: euler_bend
    links:
      - from: <instance>:<pin>
        to: <instance>:<pin>
        xsection: strip
        family: optical
        width: 0.45
        radius: 10
        routing_type: euler_bend
```

The saved `xsection` field is the bridge from frontend route editing into
`mxpic_router` and Nazca.

## Build-Time Technology and PDK Loading

Both SVG preview and final GDS now use the router-backed path.

### SVG Preview

Compact path:

`POST /api/save-layout` -> `save_layout` (`backend/server.py` line 715)
-> `create_routed_layout_svg` (`backend/routed_layout_preview.py` line 14)
-> `mxpic_router.build_project_gds` -> temporary GDS -> `gdstk.write_svg`.

Arguments passed from `save_layout`:

- `pdk_root=current_pdk_root()` (`backend/server.py` line 737).
- `technology_manifest_path=technology_manifest_path_for_project(project)`
  (`backend/server.py` line 739).
- `prefer_full_gds=prefer_full_gds_for_session(session)`
  (`backend/server.py` line 740).

The preview helper:

- Imports `mxpic_router.build_project_gds`
  (`backend/routed_layout_preview.py` line 32).
- Builds a temporary GDS with the same PDK root and technology manifest path
  (`backend/routed_layout_preview.py` lines 39-45).
- Reads that GDS with `gdstk`.
- Writes the SVG preview (`backend/routed_layout_preview.py` line 51).

### Final GDS

Compact path:

`POST /api/build-gds` -> `build_gds` (`backend/server.py` line 775)
-> `backend.gds_builder.build_project_gds` (`backend/gds_builder.py` line 26)
-> `_build_with_mxpic_router` (`backend/gds_builder.py` line 47)
-> `mxpic_router.build_project_gds`.

Arguments passed from `build_gds`:

- `current_pdk_root()` (`backend/server.py` line 789).
- `technology_manifest_path=technology_manifest_path_for_project(project)`
  (`backend/server.py` line 790).
- `prefer_full_gds=prefer_full_gds_for_session(session)`
  (`backend/server.py` line 791).

`technology_manifest_path_for_project`:

- Starts at `backend/server.py` line 183.
- Reads the project technology from `.project.json`.
- Builds:

```text
<active_role_pdk_root>/<foundry>/<technology>/technology.yml
```

- Returns the path only if it exists and stays under `current_pdk_root()`.

## Inside `mxpic_router`

Compact path:

`mxpic_router.build_project_gds` (`../mxpic_router/mxpic_router/builder.py`
line 12) -> `load_technology_manifest`
(`../mxpic_router/mxpic_router/technology.py` line 6) ->
`apply_technology_manifest` (`../mxpic_router/mxpic_router/technology.py`
line 13) -> Nazca `nd.add_layer`, `nd.add_xsection`,
`nd.add_layer2xsection`.

Detailed path:

1. Build starts:
   - `build_project_gds` starts at
     `../mxpic_router/mxpic_router/builder.py` line 12.
   - It imports Nazca as `nd`.
   - It loads the manifest from `technology_manifest_path`
     (`../mxpic_router/mxpic_router/builder.py` line 23).
   - It applies the manifest to Nazca
     (`../mxpic_router/mxpic_router/builder.py` line 24).

2. Technology manifest to Nazca:
   - `load_technology_manifest` returns `{}` if the path is missing
     (`../mxpic_router/mxpic_router/technology.py` line 6).
   - `apply_technology_manifest` does nothing for an empty manifest.
   - For each `manifest.layers` entry it calls:

```python
nd.add_layer(name=name, layer=(layer, datatype), overwrite=True)
```

   - For each `manifest.xsections` entry it calls:

```python
nd.add_xsection(name=xsection)
nd.add_layer2xsection(...)
```

   - Supported layer binding keys are:
     - `growx`
     - `growy`
     - `leftedge`
     - `rightedge`

3. Project YAML parsing:
   - `_load_project_specs` reads saved `.yml` / `.yaml` files
     (`../mxpic_router/mxpic_router/builder.py` line 46).
   - `load_cell_spec` reads YAML
     (`../mxpic_router/mxpic_router/eda_loader.py` line 80).
   - `parse_cell_dict` converts dictionaries into `CellSpec`
     (`../mxpic_router/mxpic_router/eda_loader.py` line 85).

4. Link xsections:
   - `LinkSpec.xsection` defaults to `strip`
     (`../mxpic_router/mxpic_router/eda_loader.py` line 51).
   - `BundleSpec.xsection` defaults to `strip`
     (`../mxpic_router/mxpic_router/eda_loader.py` line 66).
   - Bundle-level `xsection` is read at
     `../mxpic_router/mxpic_router/eda_loader.py` line 137.
   - Link-level `xsection` overrides bundle-level xsection at
     `../mxpic_router/mxpic_router/eda_loader.py` line 142.
   - If `family` is missing, `_family_from_xsection` maps metal-like xsections
     to `electrical`, otherwise `optical`
     (`../mxpic_router/mxpic_router/eda_loader.py` line 171).

5. PDK asset resolution:
   - `_resolve_pdk_asset` starts at
     `../mxpic_router/mxpic_router/builder.py` line 509.
   - It first tries the saved component path directly under `pdk_root`.
   - If no direct directory exists, it scans the whole PDK root by component
     folder basename.
   - It reads the component YAML if present.
   - It chooses GDS through `_find_gds`
     (`../mxpic_router/mxpic_router/builder.py` line 533).
   - Normal users prefer `<component>_BB.gds`; manager sessions can prefer
     `<component>.gds`.

6. Route generation:
   - `_route_link` starts at
     `../mxpic_router/mxpic_router/builder.py` line 269.
   - It creates the route object with:

```python
Route(radius=link.radius or 10, width=link.width, xs=link.xsection, PCB=...)
```

   - Optical and electrical routes both use `link.xsection`; metal routes also
     enable PCB-style routing for `metal_1` and `metal_2`.
   - Nazca can generate geometry on the correct layers because
     `apply_technology_manifest` already registered the xsection layer bindings.

## Current Fallbacks and Observations

- If a project has no saved technology, the frontend uses
  `FALLBACK_TECHNOLOGY_MANIFEST`.
- If `/api/technologies/<foundry>/<technology>/manifest` fails, the frontend
  also uses `FALLBACK_TECHNOLOGY_MANIFEST`.
- If build-time `technology_manifest_path_for_project` cannot find
  `technology.yml`, `mxpic_router` receives no manifest path and Nazca
  registration is skipped.
- The library browser is project-scoped by selected technology, but preview and
  GDS build pass the base role PDK root to `mxpic_router`.
- This works because normal saved component paths include
  `foundry/technology/...` relative to the base role PDK root.
- Manually edited YAML with only a component folder name can still resolve, but
  the router will scan the whole active PDK root and use the first matching
  folder basename.
- The legacy local GDS/SVG preview builder and local PDK GDS registry have been
  removed from `mxpic_EDA`; active build and preview actions call the
  router-backed API wrappers instead.