Sample Assignment Workflow - Responsiveness and Privilege Review

Large-scale document reviews often require multiple layers of review, in which documents that complete one layer of review route into the next. Linking multiple assignment groups, as described in this article, lets you automatically route documents all the way through review, including quality control (QC) in preparation for production. 

Setting up linked assignments at the beginning of your case can prevent document logjams from piling up while waiting for someone to route them. With linked assignments, documents flow into their next review based on the review work applied in the previous one.

This article gives an example of routing documents through both responsiveness review and privilege review, and incorporates two layers of QC review. This example can help you plan your own responsiveness and privilege review, or be applied more generally to any multi-layer review.

You'll use the following Everlaw tools:

• Assignment Groups
• Codes
• Binders

Requirements

Required permission: To create an assignment group, you must have Create or Admin permission for Assignments, or be a Project Admin

Before you start, make sure a Project Admin:

• Sets up the codes and other review work necessary for the assignment groups. For more information, see our Administer Codes for Review article.
• Adds all the reviewers into the project, with the appropriate permissions. For more information, see our Add and Remove Users from Projects article.

Example scenarios

Linked assignments are helpful for any multi-phase review. For example:

• When you need to produce documents responsive to specific criteria, while withholding privileged documents. Linking assignments for this purpose allows reviewers to focus on specific aspects of review, while allowing documents to be automatically routed through a multi-tiered review. This is the flow outlined in this article.
• When you have received documents and need to identify the most important ones for a litigating attorney. A subset of documents from a first pass review can be automatically routed to a second pass review for QC. 

 Scenario for this article

The example below shows how to link review phases using assignment groups for a review in response to a request for production. For a given custodian, you need to produce the responsive documents while withholding the privileged ones. In this scenario, your team reviews documents from a particular custodian, first for responsiveness, then for privilege, while also sampling non-responsive documents to evaluate first-pass responsiveness review quality. All responsive documents then get routed to a final QC review before they're considered reviewed and ready for production.

The steps outlined in the sections below can be used as general guidance for planning out linked review groups.

Step 1: Initial Responsiveness review

Create the initial assignment group for first-pass responsiveness review. This assignment group includes the full set of documents you plan to pass through review. When they're done with this review, they will get routed to either the next phase (Privilege review), or be sampled for a QC review to check the quality of the initial coding. 

In this specific example the review is for documents from custodian John Smith, but a general best practice for the initial review is to define the assignment group to include the complete set of documents intended for review.

Inclusion criteria: Documents for Custodian: John Smith

Review criteria: Any document that meets both the inclusion criteria and review criteria will count as reviewed for this assignment group. In this example, the review criteria should be Coded: under Responsiveness. This means that a document will count as reviewed once any code from the Responsiveness category has been applied.

Dynamic vs. Static: Dynamic groups update as new documents meet the inclusion criteria. If you expect new documents from this custodian to get added into the project and want them incorporated into the assignment group automatically, set the group to be Dynamic so that they are automatically added into the group.

Step 2: QC Responsiveness review

This QC phase helps you assess the reliability of first-pass review. If this review results in a lot of overturned coding, you might consider a more robust QC process or additional guidance for your first group of reviewers. If only a few documents appear mis-coded, you may decide targeted correction is sufficient.

For this QC, the second assignment group captures documents from the first phase that were coded Not Responsive. This assignment group is used to assign a subset of those documents to a second team for QC sampling. 

Reviewers for this phase can get started as soon as documents start flowing out of the initial responsiveness review.

Inclusion criteria: For a QC review, the inclusion criteria should reference the review criteria of the initial review. In this example, the QC is specifically for the documents initially coded as Not Responsive, so the inclusion criteria should be set to include documents Coded: Responsiveness: Not Responsive. 

If you want only a subset of these documents reviewed, you can use Search settings to set a sample size. This lets you pre-determine how many documents will be reviewed in this pass.  

Review criteria: When linking assignments, make sure the downstream assignment’s review criteria do not treat incoming documents as already reviewed. In this case, this QC assignment is considering review work that is already applied, so you must add an additional review criteria to indicate that the documents have been through this pass. Do this using one of the following two strategies, which are described in more detail in our Linking Assignments Together article:

• Strategy 1: Make sure that the reviewers for this phase are in a different user group from those who did the initial review. The review criteria should include both a code from the responsiveness category and the Viewed term specifying that user group. For example, if your reviewers for this phase are those in the Second Pass Review user group, your review criteria would be: Coded: under Responsiveness AND Viewed: Second Pass Review.
  

  • Adding the Viewed term ensures that the document will count as reviewed once someone from this group has reviewed it, regardless of whether they overturn the coding or not.

    Note

    You can also choose specific people for the Viewed by term. Make sure you connect them with an OR so that a document counts as reviewed as long as it is viewed by any one of the selected reviewers.

  • Allowing the Coded term to be any code from the Responsiveness category means that documents whose coding is overturned from Not Responsive to Responsive will still count as reviewed 
• Strategy 2: Require that, in addition to having a Responsiveness code applied, reviewers apply a specific code to indicate that documents have been reviewed for this phase. The Review Criteria would be Coded: under Responsiveness AND Coded: Second Pass: Complete
  
  This is a good option if you have overlapping reviewers for the two phases and can't use the Viewed term to segregate them.

Assign to users: If you are using the Viewed term for your inclusion criteria, make sure you select the same user group or individual users within that user group as those who get assigned these documents. In this example, it would be the Second Pass Review Group.  

Step 3: Privilege review of responsive documents

Create another linked assignment group for privilege review. In the sample workflow, this group receives documents coded responsive in the Responsiveness review, as well as documents from the non-responsive sample assignment whose coding was changed to responsive in QC. The purpose of the privilege review is to assign codes indicating privilege status to responsive documents. Privilege status will determine which documents are ultimately withheld from being produced, despite meeting the criteria for being responsive.

Reviewers for this phase can get started as soon as documents start flowing out of the initial responsiveness review.

This assignment should be Dynamic so that documents automatically enter the group as review decisions in earlier phases make them eligible for this phase. 

Inclusion criteria: Coded: Responsiveness: Responsive

Review criteria: The review criteria for this review adds a privilege code to the Responsiveness code that is already applied. When documents finish this phase, they should have both a code from the Privilege category and the Responsive code applied.

Step 4: QC review

After privilege review, create a final assignment group for pre-production QC. This phase captures reviewed documents from the privilege-review assignment and routes them to a final team for checks such as coding accuracy, family completeness, and binder placement.

We recommend that the review criteria for this assignment includes adding documents to a specific "To be Produced" binder, which can then be used as the production criteria when you create a production protocol. This means that this final QC review will populate the binder with the documents ready for production. Documents that are found to be non-responsive during QC review will not be added to the production binder. 

Pre-production QC is a recommended last step before production, to make sure you're sending out the right set of documents.

Inclusion criteria: Documents Coded Responsiveness: Responsive and Coded: under Privilege

This defines all the documents that have been identified as responsive and have been reviewed for privilege. These are the documents that need QC before being produced.

Review criteria: Depending on QC, these documents might end up in the To be Produced binder, or have their Responsive code overturned to indicate that they are not responsive and therefore not intended for production. A document that has been through this phase of review needs to meet either or these two criteria:

• Be in the Ready for production binder OR

• Coded Responsiveness: Not Responsive AND viewed by a member of the QC reviewer group. 

  Note

  Adding the Viewed term distinguishes the non-responsive documents coming out of QC from those reviewed not responsive in the first pass of review. If you have overlapping viewers between the earlier passes and QC step, you can instead require that reviewers add a QC: Completed code to all documents they review in this phase.

Conclusion and next steps

When you create these assignment groups, you create a logical flow of documents to move through the review. The guidance on creating multi-level review via linked assignment groups is generalizable across review types, even if the specific review and inclusion criteria vary. 

As next steps, we recommend that you make sure your reviewers understand how to use assignments and apply review work. You can send them our Receiving an Assignment article and How to Code a Document in the Review Window and Use Coding Presets article.

You can use our Modify and Manage an Assignment Group article to learn more about managing in-progress assignments.