Batching (updated draft)

Tenant and project administrators can create batch sets of documents for distribution to reviewers. Batch review supports two types of workflows, three types of document batches, and a number of advanced features that assist the review team in managing, tracking, and assuring a high quality data mining review.

Batching information can be viewed from the dropdown under the Review heading:

Batching actions can be accessed from the kebab menu at the top right of the Documents page:

There are three different types of batches defined in Canopy:

Review

Review batches are the standard, first-level batch type for distributing documents to the review team.

Alternative Workflow (“Alt Workflow”)

Alternative Workflow batches contain documents that are complex and require a specialized skill to review. Alt Workflow documents are often reviewed by the Assessment team. Assessors can batch structured data, like Excel spreadsheets or databases, directly to an Alt Workflow batch, bypassing the Review batch. Alt Workflow batches can also be created by administrators when reviewers encounter trouble reviewing certain documents, for example a large PDF report.

Quality Assurance (“QA”)

Quality Assurance batches are meant to be used by either the Assessment or Review teams to perform cross-team QA checks or QA check their own work.

Batch Sets

Multiple batch sets can be created for any of the three batch types.

The project team can choose to either configure their batch types to adhere to Canopy’s Default Strict or Standard workflow rules.

Canopy’s default workflow incorporates review batches, alternative workflow batches, and quality assurance batches into a single review process.

Workflow:

1. Users can tag documents for Alt Workflow via different methods:
   • Project admins or review managers can tag documents for Alt Work from the document view.
   • Reviewers can apply an Alt Workflow tag while reviewing a batch of documents.
   • The system automatically applies the “Bulk Mapped” Alt Workflow tag to bulk-mapped documents.
2. Project admins or review managers can create Alt Workflow batches from documents with an Alt Workflow tag.
3. Project admins or review managers can create Review batches from documents tagged Alt Workflow or documents not batched.
4. Project admins or review managers can re-batch QA rejected documents to their respective review batch type: Review or Alt Workflow.
5. Once batched, a document can only appear in one Pending batch at a given time.
6. Project admins or review managers can create QA batches from documents in completed Review or Alt Workflow batches.
7. The review process is completed for a document when a user marks a document as “approved” in a QA batch.
8. Users can only re-batch QA-approved documents if they delete the batches.

    stateDiagram
        bulk_mapped : Admin Bulk Mapped
        not_batched: Documents Not Batched
        reviewed_rvw: Completed RVW Batch
        pending_rvw: Pending RVW Batch
        tagged_awf: Documents Tagged AWF
        pending_awf: Pending AWF Batch
        marked_awf: Completed AWF Batch
        pending_qa: Pending QA Batch 
        rejected_qa: Rejected QA Documents
        approved_qa: Approved QA Documents
        pending_qa_awf : Pending AWF QA Batch 
        rejected_qa_awf : Rejected QA AWF Documents
        review: Review
        awf: Alt Workflow
        available_rvw : Available
        in_progress_rvw : In Progress
        available_qa_rvw : Available
        in_progress_qa_rvw : In Progress
        available_awf : Available
        in_progress_awf : In Progress
        available_qa_awf : Available
        in_progress_qa_awf : In Progress
        auto_tag : AWF Tag "Bulk Mapped"
        auto_create_batch : Auto Batched AWF
        auto_complete_batch : Auto Completed Batch
                        
        [*] --> not_batched
        not_batched --> review : Batched for Review
        not_batched --> tagged_awf : Admin Tagged Documents AWF 
        not_batched --> bulk_mapped : Admin Bulk Mapped Documents
        state review {
            [*] --> pending_rvw
            state pending_rvw {
               available_rvw --> in_progress_rvw 
            }
            pending_rvw --> reviewed_rvw : Documents Marked Reviewed
            reviewed_rvw --> pending_qa : Batched for QA
            state pending_qa {
               available_qa_rvw --> in_progress_qa_rvw 
            }           
            pending_qa --> rejected_qa: Not Approved
            rejected_qa --> pending_rvw : Rebatched Documents
        }
        tagged_awf --> awf
        state bulk_mapped {
                 auto_tag --> auto_create_batch
                 auto_create_batch --> auto_complete_batch
         }
        note right of bulk_mapped
            Bulk mapped documents are <br>swept into AWF QA batches.
        end note    
        state awf {
            [*] --> pending_awf : Batched for AWF
            
            state pending_awf {
               available_awf --> in_progress_awf 
            } 
            pending_awf --> marked_awf : Documents Marked Reviewed
            marked_awf --> pending_qa_awf : Batched for QA
            state pending_qa_awf {
               available_qa_awf --> in_progress_qa_awf 
            } 
            rejected_qa_awf--> pending_awf : Rebatched Documents
            pending_qa_awf --> rejected_qa_awf : Not Approved
        }
        review --> tagged_awf: Reviewer Tagged Documents AWF 
        review --> approved_qa: Approved Documents
        awf --> approved_qa: Approved Documents
        approved_qa --> [*] : Reviewed Documents
        
        

Removed Restrictions

First Review

1. Batch all documents in either RVW or AWF
2. Complete some batches
3. Partially complete some batches

Second Review (Simultaneous)

1. Create an AWF Batch Set of some documents from both AWF and RVW

• Include Completed Documents from First Review
• Include Not Reviewed Documents from First Review

4. Review these docs

Third Review (Simultaneous)

1. Create an AWF Batch Set of some documents from both AWF and RVW

• Include Completed Documents from First Review
• Include Not Reviewed Documents from First Review

4. Review these docs

No Batch Actions:

• Admin QA
• Alt Workflow
• Review -> AWF -> QA

Administrators can remove restrictions that will allow for the following in order to batch documents to multiple batch types simultaneously.
To switch to the standard workflow option, administrators can navigate to Settings->Project->General->Manage Review Setting and disable the “Enforce Batching Workflow Restrictions” and click on the toggle to remove batching workflow restrictions. Taking this action to disable the strict workflow will alert administrators that they will be unable to return to the strict workflow rules after confirming removal of restrictions.

1. Documents can only be present once per each combination of batch type and batch set.
2. Repeat review batches can only be created from existing batches of other batch types.

In the standard review workflow, the strict rules are replaced by guidance that provides feedback on where the documents chosen to be batched already exist in other batch sets:

Delete Batches in Non-Restrictive Worklow

Deleting batches from any batch type will remove the same document from all batch types. For example, if document 123456789 exists but the review team has not reviewed it in both a review and an alt workflow batch, and the user chooses to remove the AWF batch, the system will remove the document from both batches.

    stateDiagram
        not_batched: Not Batched
        reviewed_rvw: Reviewed RVW
        pending_rvw: Pending RVW
        tagged_awf: Tagged AWF
        pending_awf: Pending AWF
        marked_awf: Reviewed AWF
        pending_qa: Pending QA
        rejected_qa: Rejected QA
        pending_sa: Pending QA
        rejected_sa: Rejected QA

        approved_qa: Approved QA
        pending_qa_awf : Pending AWF QA 
        rejected_qa_awf : Rejected QA AWF
        review: Review
        awf: Alt Workflow
        qa: Standalone QA
        repeat_review: Batch Repeat Review
        repeat_awf: Batch Repeat AWF
        repeat_qa: Batch Repeat QA
        
        review  --> repeat_qa: New QA Batch
        review  --> repeat_awf: New AWF Batch
        awf  --> repeat_review: New Review Batch
        awf  --> repeat_qa: New QA Batch
        qa  --> repeat_review: New Review Batch
        qa  --> repeat_awf: New AWF Batch
        repeat_review --> review
        repeat_qa --> qa
        repeat_awf --> awf
        
        [*] --> not_batched
        not_batched --> review : Batched for Review
        not_batched --> tagged_awf : Tagged AWF 
        not_batched --> qa : Batch for QA 
        not_batched --> awf : Batch for AWF
        state review {
            [*] --> pending_rvw
            pending_rvw --> reviewed_rvw : Marked Reviewed
            reviewed_rvw --> pending_qa : Batched for QA
            pending_qa --> rejected_qa: Not Approved
            rejected_qa --> pending_rvw : Rebatched 
        }
        tagged_awf --> awf
        state awf {
            [*] --> pending_awf : Batched for AWF
            pending_awf --> marked_awf : Reviewed AWF
            marked_awf --> pending_qa_awf : Pending AWF QA
            rejected_qa_awf--> pending_awf : Rebatched
            pending_qa_awf --> rejected_qa_awf : Not Approved
        }
        state qa {
            [*] --> pending_sa
            pending_sa --> rejected_sa
            rejected_sa --> pending_sa
        }
        
        review --> tagged_awf: Tagged AWF
        review --> approved_qa: Approved Document
        qa --> approved_qa: Approved Document
        awf --> approved_qa: Approved Document
        approved_qa --> [*] : Reviewed Document
        
              

The permission to create batch sets is enabled for administrators. After clicking on the kebab menu at the top right of the Documents page and selecting Create Batch Set, all the documents filtered in the document list are considered in scope for creating a batch set.

Administrators can create new Review, Alt Workflow, or QA batch sets based on their selected criteria, or add documents to an existing batch set:

The “sweep families” feature is available for Review and QA batch types. If selected, any documents that are eligible, based on the workflow criteria, will be checked for family members. If family members are found, and the family member is eligible per the workflow criteria, the family member will “swept” into the batch:

The option to include documents with similar file names is available for Review and QA batch types. If selected, any documents that are eligible, based on the workflow criteria, will be checked for file name similarity. If similarity is found, the entire family will be batched. If similarly named documents meeting the workflow criteria are found, they will also be batched:

The “Ignore duplicates” option is available for all batch types. Selecting this option will skip batching duplicates:

The option to create a new batch set is available for all batch types. Review managers have the option to create a new batch set or add to an existing batch set. Mandatory fields include:

Batch Set Name

The name of the batch set

Batch Set Prefix

The prefix to use in naming the batch

Batch Size

The number of documents to include in each batch

The scope option is available to fine tune the document selection when batching documents for QA.
The administrator or review manager can choose to include all reviewed documents from all batch sets or only reviewed documents from specific batch sets:

Only tenant or project administrators and review managers can create Alt Workflow (AWF) tags. Available AWF tags that have been created can be used by reviewers and lite reviewers. The Alt Workflow button at the top of the Document Info page can be clicked to apply an existing AWF tag to the document or a new AWF tag can be created:

When a document is tagged with an AWF tag, the Mark Reviewed button will disappear and the document will move into the Reviewed tab in the batch.

• All tenant and project administrator, review manager, reviewer and lite reviewer roles in the project are able to access the batches.
• Only one user is allowed to take one In Progress batch at a time.
• Tenant or project administrators and review managers can view batches in the Documents page via a button, found in the Batches dropdown under the Review heading:

After a user checks out a batch, the review work can begin.

• When reviewers or lite reviewers in non-administrative roles check out batches, the batches will move to the In Progress tab and will then be viewable on the Documents page.
• Reviewers and lite reviewers do not see batches that have been checked out by tenant or project administrators, review managers, or other reviewers, however tenant and project administrators and review managers can see everyone who has accessed the batches.
• Reviewers and lite reviewers cannot unassign a batch accessed by themselves or accessed by tenant or project administrators or other reviewers.
• For In Progress batches, users can check in only their accessed batch. Administrators cannot check in batches accessed by reviewers or lite reviewers.
• On the document list for the batch, the option to mark documents as Not Reportable will appear after selecting one or more documents:

If entities are detected in documents selected to be marked as Not Reportable, the following warning will appear so reviewers can choose to skip marking Not Reportable on documents with detected entities:

Skipped documents will remain in the Pending status and documents marked as Not Reportable will be marked as Reviewed.

The Document Info page displays details, including the type of batch, batch name, and buttons to mark documents as Reviewed, Not Reportable, or Alt Workflow. The reviewer’s name and email ID will also be displayed in the left panel after the document has been reviewed:

Tenant or project administrators and review managers can unassign users from that batch by clicking the “x” icon beside the name of that user under the In Progress batch tab:

Tenant or project administrators and review managers can split large batch sets into smaller batches by clicking on the Split Batch icon:

Administrators are able to remove documents marked “Reviewed” from batches. A confirmation request will appear and administrators can use the radio buttons to select removal of documents marked “Reviewed” from the current batch or from all batches associated with the project:

Tenant or project administrators and review managers can delete batches using the trash can icon. If the batches contain fully or partially reviewed documents, users will be asked to either keep these documents in their batches, in which case the review work is saved, or remove them from the batches, in which case the review work is not saved:

If the Continue button is clicked when the first radio button selected, as pictured above, the batch set is removed from the list and an alert is displayed to explain that a batch set containing partially reviewed or reviewed documents cannot be deleted: