December 18, 2020 Release: Clustering (beta), upload PDFs without a load file, and more!

Expected US release: 18 December 2020

Expected AU, CA, EU, and UK release: 28 December 2020

This substantial release adds many new and exciting features to all parts of the Everlaw platform! First, you will be able to visualize documents by conceptual similarity with Clustering. Second, you can now upload PDFs without a load file. Third, leverage an Everlaw Application Programing Interface to create custom applications. Fourth, now process and produce your documents faster with significant speed improvements. Finally, you can now duplicate the granularity of the search page on all other search builders on the site. Read on to find out more about the features coming out this month!

Improvements:

_____

Clustering (beta)

Clustering has been released in beta with limited customer availability. As you can imagine, evaluating the billions of documents on the platform to determine their position in a cluster takes time and we want to be sure that we do it right without disrupting your existing workflows. If you are interested in using Clustering immediately and providing feedback about your experience, please reach out to support@everlaw.com. 

Clustering visualizes documents in your dataset by conceptual similarity. It generates insights about concepts in documents without requiring any user input. Traditional search tools require you to have a baseline understanding of what’s in your documents and what to search for, but with Clustering, you can conduct ECA and begin to understand your dataset without any prior knowledge of the contents of the documents. 

On the Clustering page, an individual document is represented as a data point. Each document belongs to a color-corresponding cluster. The cluster shape is an approximation based on document densities and similarities within that cluster. Terms associated with each cluster are generated by the clustering algorithm and deemed as exemplar. The terms can give you a sense of the concepts within the documents.

To explore your data, you can zoom in and out, hide and show documents or clusters, filter your documents with a search query, and compare document properties like metadata and data type with Data Visualizer. 

If you are responsible for assessing the quality of review decisions made by your team, you can utilize a coding overlay to identify potentially uncoded or incorrectly coded documents within the same cluster as similar documents. You can select the coding category of interest from a dropdown list, identify outlier documents by color, and see whether documents that are conceptually similar are coded differently. The overlay tool also allows you to color code by prediction score and rating, unlocking both exploratory and review administration workflows. 

CLUSTERING_FINAL.gif

Learn more about Clustering. We’ll also be covering Clustering in our new Visualizing Your Data training session.

Upload PDFs without a load file

Everlaw’s processed uploader now allows for a seamless and convenient upload for produced PDFs without a load file. Simply drag and drop a folder of PDFs named after Bates numbers, and Everlaw will upload the documents with the detected Bates number as processed data, alongside extracted text and metadata.

The uploader accepts documents that are named with or without their Bates numbers. Everlaw gives users an option to upload documents by their detected Bates numbers, such as uploading ABC001.pdf as document ABC001 on the platform. All other PDFs that do not have a detected Bates number will be uploaded with a default prefix EVER. 

If you have database upload permissions, you can find the new option to upload PDFs without a load file in the Processed Data tab of a project and upload up to 5000 PDFs at a time.

upload_pdfs_FINAL.gif

Everlaw’s processed uploader reads the filename of the PDFs to detect its Bates prefix and number. Once you have verified the prefixes of your upload folder, the PDFs will then be uploaded accordingly with extracted text and Filename and Filepath metadata. 

Learn more about uploading PDFs without a load file. 

[Organization Admin] Everlaw API

You can now build custom applications leveraging Everlaw’s Application Programming Interface (API). This first version lets programs access Everlaw project management information like database sizes, permission groups, user information, and more. Everlaw_API.gif

To generate API keys, use the new API Keys tab of the Organization Admin dashboard. API keys allow programs using Everlaw’s API to securely connect to your organization’s data. 

Learn more about the Everlaw API in this article.

Faster processing speeds

Improvements to our processing pipelines now allow you to ingest and produce documents at up to 900,000 documents an hour, an increase of 80% from prior speeds.

Expanded search terms

When creating searches outside of the search page—for example, when creating assignment groups, creating productions, adding conditional coding rules, or building a predictive coding model—there are three major improvements:

  • Access additional search terms: many Document and Review search terms previously only available on the main search page have been added to these other platform searches.
  • Access common search terms with one click, now ‘pinned’ to each search interface. 
  • Navigate search builders across the site with keyboard shortcuts. The shortcuts are the same as those found on the main search page; press Shift + “?” to view the list.

expanded_search_terms_-_create_assignment_group.png

Preserved formatting when importing to Storybuilder

We have added more support for document formatting into Storybuilder imports. Hyperlinks, text justification, and empty characters in your DOCX files will now be properly imported when imported into Drafts and Depositions. 

Bates type metadata

Non-standard metadata fields with Bates values can now be classified as type Bates instead of type Text in the processed uploader. You will be able to search over these fields by prefix and by range, instead of just by text match.

bates_type_search.gif

Bates type metadata fields can be made editable by project administrators in Project Settings. To edit a Bates type metadata field, you must include both a prefix and a number. 

edit_bates_type.gif

Search by number of attachments

In complete projects, you will now see the option to search by a new field, Attachment Group Size. The Attachment Group Size represents the total number of attachments in an attachment family. Thus, if an email has three attachments, both the email and the attachments will have an Attachment Group Size of 3.

attachment_group_size_100.gif

Attachment Group Size can also be added as a column in the results table.

attachment_group_size_results_table_column.png

Batch delete redaction instances

You can now delete redactions in bulk. To do so, you should select the documents you wish to include in your batch deletion from the results table, then click on the Batch dropdown in the toolbar and select Redact.

batch_delete_redactions_panel_100.gif

The redaction panel will now have a “Delete” toggle. You can click on this toggle to access the redaction deletion panel. From here, you can search for redacted words or phrases via text search. The batch redaction panel will automatically display sample hits for all text redactions.

batch_delete_terms_100.gif

Note that if you search for redacted content via text search, we will only remove text redactions that exactly match your search query. Image and full-page redactions will not be removed.

You can also delete redactions on a particular metadata field or all redactions with a particular stamp.

After you confirm your batch redaction removal, a new card will be created in the Batches & Export column. You can view and download the details of your batch redaction deletion here, as well as undo the deletion.

Increased hit highlights

PDF viewer hit highlighting limits will be increased to approximately 2500 pages/10MB of text. Previously, the limit was 1000 pages/5MB.

Accessibility improvements

You can now access many new clickable elements across the site via keyboard. The tab key and shift + tab keys now allow you to move between many clickable buttons, icons, cards, and pages on the site. The up and down arrow keys can also be used to navigate between menu items. You can press the enter key to activate most keyboard-focused elements or press the spacebar to select checkboxes and radio buttons. Lastly, you can now exit out of dialogs without action by pressing the tab key and the esc key.

Learn more about Everlaw’s accessibility initiatives.

Testimony text in CSV exports

In Storybuilder Timeline, you can now include the testimony text as a field in a CSV export. By clicking Export and selecting CSV, you can choose from a list of options to export as fields. This will now include “Testimony text.” 

export_csv.png

Learn more about how you can preserve meaningful testimony from Deposition in these help articles.

Search for auto-coded documents

In the Auto-Code Rules page in Project Settings, you can now view and refine documents that have been auto-coded. By clicking the document count in the auto-code rules table, you can launch a results table of all documents currently auto-coded by a particular rule. Then, you can click the refine icon in the results table. This opens the search query with an Auto-Code Rule search term, where you can specify results by people and time for one or multiple auto-code rules.

autocode_gif.gif

Learn more about applying and navigating auto-code rules.

Edit metadata time fields

You will now be able to add times to editable metadata fields with type DateTime. Likewise, you will be able to add times to freeform codes with type DateTime. 

freeform_datetime_100.gif

When you have entered a time, you will see an option to select a timezone. If a new timezone is selected, the freeform code or metadata field will translate the time in the project timezone. You can hover over or click on the freeform code or metadata field to see the time in its original timezone.

freeform_timezone_100.gif

Customize date and time metadata display

You can now choose how you would like your date and time metadata to display. By default, the date and time formats are set to the regional norm of your VPC. You can adjust the date or time format in the Date and Time Format section of the General tab in Project Settings. This setting affects how Everlaw date/time metadata fields are displayed, Story dates/times, and freeform codes with the DateTime format. 

date_and_time.png

Produce placeholder text in text files

You can now produce placeholder text files for withheld documents. By default, productions will continue to include no text files for withheld documents. If you wish to produce placeholder text matching your image placeholders, use the new advanced settings dropdown of the withholding or privilege rules step of the production wizard. 

protocol_placeholders.gif

Learn more about production protocols.