Database Mapping Design
Link to Database Mapper Prototype
flowchart TD
Identify[Identify Database Files] --> Select[Select Database]
Select --> Load[Load Database in SQL and Report Size]
Load --> Decision{Start Project?}
Decision --> |Yes| Sample[Sample Data]
Decision --> |No| Delete(((Delete Database)))
Sample --> Explore[Explore with Samples]
Explore --> mappable_sg
Explore --> freeform_sg
Explore --> json_sg
Explore --> docs_sg
subgraph mappable_sg [Identify Mappable Fields]
Relate[Identify Mappable Fields] --> Link[Link Fields]
Link --> View[Create View]
View --> Export[Export Full Data View]
View --> |Revise| Relate
end
subgraph freeform_sg [Identify Free Form Fields]
Freeform[Identify Free Form Text Fields] --> FullFree[Export Freeform Field As CSV ]
FullFree --> ProcessFree[Process and detect on CSV]
end
subgraph json_sg [Identify JSON Fields]
Json[Identify JSON Fields] --> |Predictable| LinkJson[Link Json Fields]
Json --> |Unpredictable| FullJson[Extract Field Entries as seperate docs]
FullJson --> ProcessJson[Process JSON Documents]
ProcessJson --> DetectJson[Run Detection on JSON Documents]
end
subgraph docs_sg [Identify Doc Link Fields]
Document[Identify Fields Linked to Docs] --> Fetch[Fetch and Process Documents]
Fetch --> Key[Select Fields to Search and Add As Entities]
end
Key --> Map
LinkJson --> Map
Export --> |Approved| Map[Map Fields to Entity List]
ProcessFree --> Datamine[Data Mine]
DetectJson --> Datamine
Key --> Datamine
Map --> Consolidation
Datamine --> Review[Review Documents]
Review --> Consolidation(((Run Consolidation)))
Selecting Database List from the Review menu lists all the SQL Server and MS Access Databases uploaded in the review population.
Once you are on the database list view you can perform the following actions:
- View Schemas
- Go to the read only schema view.
- Configuring Mapping
- Configure and finalize one or more views of PII.
- Map Entities (can be bulk action)
- Only, selectable if all views are already mapped, finalize, or marked for deletion. Will map all entities for finalized views and unmap entities for views marked for deletion.
- Common Table Filter
- Filter/search common component
- Select one or more databases to “Map Entities”
- Fields
- Default:
- File Name
- Data Base Name
- Size of File
- Number of Schemas
- Number of Tables
- Number of Views
- Draft
- Ready to Map (Final)
- Ready to Unmap (Delete)
- Mapped
- Selectable:
- SHA256
- SHA1
- MD5
- Modified Date
- Creation Date
- etc…
- Default:
We believe mapping will take awhile. We will need to alert them to this effect. Ideally our progress bar would give them an estimated time.
This is the screen you will arrive at if you select view schemas from the list view. This is a view only screen that will allow you to explore the database structure.
Once you are on the database list view you can perform the following actions:
- Select Schemas
- Allows the user to select the schema they would want to explore.
- View Sample Table Data
- The user would be able to click on the table to pop up a table full of randomly sampled data.
When creating a new view you will arrive at this screen.
- Create View
- Create new draft view
- Edit View
- Edit an existing view
- Copy View (active if the view is mapped)
- Copy and Edit Existing View
- Delete View (active if the view is mapped)
- Mark view to be deleted and unmapped
- View Entities (active if view is mapped)
- View Mapped Entities -> brings you to a filtered raw entity view (only will work if entities are currently mapped)
While mapping and unmapping actions other than view only are disabled. No changes can be made to views during this time.
- Create View (Should this really be here?)
- Configure a new view
- Save Draft
- Save a draft of the current view I am working on.
- Finalize
- Finalize the current view I am working on by validating the view and setting the status to ready to map.
- Success - popup showing list of attributes and their associated PII field
- Failure - popup showing list of issues.
- Validate
- Validates view and points out anything the needs to be completed before you can finalize. Validates no conflicting relationships, no unrelated tables, all attributes associated with a field.
- Select VIew (Should this really be here)
- Select a view to edit from the left hand side.
- Select one or more schemas
- Select one of more tables
- View Sample Table Data
- Delete Relationships
- Create Relationships
- Select Attributes
- Show original Relationships
- Associate Attributes with PII Fields
Please add total row counts to each table Please add join inner, outer, left, right to the relationship config. We need to show that we will recommend mapping/PII associations
We should add “Randomly Sampled”
