Linking Assignments Together

Everlaw’s assignment group inclusion criteria allows you to link one assignment to a second assignment, which can be used to achieve a variety of workflows. A common use case is for a second pass-review,  in which documents in the first assignment that meet a certain review criteria are dynamically sent into a second assignment, where another group of reviewers will check the decisions of the first group.

For this video, let’s imagine that you have a first-pass review group, who will rate documents as hot, warm or cold. From there some second pass review groups will verify the decisions of the first group. One of those second pass review groups will look at all the hot and warm documents reviewed by the first group and either verify they were reviewed correctly, or change the rating to cold if they were in fact irrelevant. Now that you have an understanding of the scenario, let's walk through the steps to achieving this workflow. 

First, let’s set up our first-pass review, or the initial assessment of relevance for a set of documents. From the homepage, open the assignment group creation tool by clicking the plus sign icon next to the Assignments column header and selecting “Create New.”

Let’s say that in this case, you want to do a first-pass review of all the documents with a particular author. Using the query builder, you can very easily build a search that will return all of the documents you are looking for, and use this search as your inclusion criteria such that those documents feed directly into this assignment.

Screen_Shot_2020-02-19_at_11.17.02_AM.png

The next prompt for the assignment creation wizard is to name your Assignment group. In this example, we’ll use “First pass of Carolyn Campbell documents.”

Choose the individuals on your first-pass review team.


Screen_Shot_2020-02-19_at_11.18.44_AM.png

You can allocate by document or page count, and decide whether to make the group static or dynamic. In this example, we will choose dynamic, which means that all future documents that are uploaded into the database and have “Carolyn Campbell” as the value in the Author metadata field will automatically be included in the assignment group to be reviewed. However, you may also choose “static” if that fits your workflow. Static assignment groups do not change over time to match specific inclusion criteria.  Documents only enter the group if they are manually added. 

Finally, decide the size of your initial document allocation in the assignment group. If you leave some documents unassigned, you can also choose to let your reviewers self-assign documents from the unassigned pool. This saves reviewers from having to wait for you to manually assign them more documents when they have completed their batch. Leaving the “Batch size” box blank gives the reviewers flexibility to decide how many documents they want to claim in subsequent assignments from this assignment group.

Screen_Shot_2020-02-19_at_11.20.49_AM.png


At this point, we will set the review criteria. These are the conditions that the documents must meet to be considered reviewed. For example, this review criteria holds that to be considered reviewed, documents must be rated hot, warm, or cold. Please note that having "Rated: Any Rating" as your review criteria is sufficient for this; this search is broken up into Rated: Hot, Rated: Warm, or Rated: Cold for clarity.

Screen_Shot_2020-02-19_at_11.21.58_AM.png

Now you can create an assignment group for a second-pass review of these documents, so that others can quality check the decisions of the first-pass reviewers. The second-pass review group is only responsible for reviewing documents that were rated Hot or Warm during the first-pass review. They will verify whether the documents coded Hot or Warm by the first group are coded correctly. If they are, the ratings will remain. If they are not, they can change the rating from Hot or Warm to Cold. 

To do so, within the “Assignments” page, let’s create another assignment to link from the first assignment. Since you want this assignment to include all hot and warm documents from the first assignment, you must set up inclusion criteria to reflect this. Construct a search using the query builder that will pull in documents from your first-pass review by selecting the Assigned term. Choose your first assignment. Within that assignment, you only want the documents that received Hot or Warm ratings. Build this search, then click next. Essentially, the assignments that are filtered into the second-pass assignment group are dependent on the review decisions of the first group.

Screen_Shot_2020-02-19_at_11.23.16_AM.png

You can repeat the naming and document allocation steps taken when constructing your first-pass assignment group. However, this assignment must necessarily be “dynamic.” This is so as review progresses, all documents matching that criteria will be automatically added to the assignment group.

Screen_Shot_2020-02-19_at_11.24.34_AM.png

As a note, if the second-pass group is set up before any documents in the first-pass group have been rated, there won’t be any documents matching your inclusion criteria. Therefore, you won’t be able to allocate any documents in the final step of the assignment group creation wizard.

Now comes the trickiest part. You want your second-pass assignment group to include the same review criteria as your first-pass assignment group. Even though only hot and warm documents are coming into the second-pass assignment group, a second-pass reviewer might decide to override the first rating decision, and rate a document Cold when they come across it.

However, the review criteria of the second-pass assignment group cannot be exactly the same as the first-pass group (documents having any rating), since this would mean that all of the documents coming into the second-pass group would already be considered reviewed. Because of how assignments allocate documents in Everlaw, you cannot assign already-reviewed documents, nor can reviewers claim already-reviewed documents. As a result, an assignment group that only pulls in documents that it already considers reviewed is functionally useless.

You can add a Viewed By review criterion, such that the documents in your second-pass assignment group are only considered reviewed if they satisfy the criteria of the first-pass group, and have also been viewed by a member of the second-pass group. This second linked assignment says that all documents from the first assignment group “First pass Review of Carolyn Campbell documents” rated hot or warm, are only considered reviewed when viewed by Claire or Daniel and are given any rating: hot, warm or cold. 

Screen_Shot_2020-02-19_at_11.25.24_AM.png

Congrats! You've created an automated assignment workflow that will update dynamically as uploads and review progress. 

Have more questions? Submit a request

0 Comments

Article is closed for comments.