Document summaries, topic analysis, custom extractions, and Q&A

This article is a deep dive on the summaries, topic analysis, custom extractions, and document Q&A features, which are available as part of Everlaw AI Assistant. 

• If you're looking for a starter guide for all Everlaw AI Assistant features, see this article.
• If you're looking for a general overview of Everlaw AI assistant and information about our privacy and security standards, please see our AI Assistant FAQ. 
• If you have feedback or questions, feel free to email us at beta@everlaw.com. We love hearing from our users.

Table of contents

• Overview and background
• Document summaries
  • Chunked summaries and shorter descriptions
  • Using summaries in the Review Window
  • Generating summaries in batch and exporting summaries
  • Searching based on summaries
• Topics extraction, with integrated entity and sentiment analysis
  • Generating and viewing topics
  • Navigating topic snippets
  • Searching/filtering topics in the Review Window
  • Additional options in the Review Window
  • Generating topics in batch
  • Searching documents by topics
• Custom extractions
  • Creating an extraction request
  • Using extractions
• Document Q&A

Overview and background

Accessing the features: Document summaries, topic analysis, custom extractions, and document Q&A can be accessed from the Everlaw AI context panel in the Review Window:

• The summaries and topic tasks are located in the Overview tab
• Custom extractions is located in the Extractions tab
• Document Q&A can be accessed from the Ask questions about this document button in the footer of the panel

In addition, the batch version of summaries and topics can be accessed from the results table via the "Batch" icon in the toolbar. 

Saving results:

• Summaries and topics: These generations are saved per document, per database. This means that, for example, a summary generated for a document in one project will also be available for that document in other projects in the same database. 
• Extractions: these generations are saved per document, per project. Unlike summaries and topics, extractions are not shared across projects in a database
• Document Q&A: These generations are saved per user, per project. 

Deleting results and regenerating: From the Review Window, you can discard a saved result for a task. This means the next time the task is run against the document, a new generation request will be sent to the LLM. The substance of the result for a given task on a given document should rarely change, so you generally do not need to regenerate output, unless there are clear errors in the current result. 

Document text: If the PDF image for a document has extractable text, Everlaw uses that as the basis of analysis. If not, Everlaw falls back on the OCR text file. For batch tasks, Everlaw will always use the OCR text file. Documents that do not have extractable text or an OCR file cannot be analyzed by the Everlaw AI Assistant. 

Document length limits: Everlaw currently limits the length of supportable document text to approximately 300 “pages” (a page being defined as ~500 words). If you run a task against a document that exceeds this threshold, we will let you know, via a banner in the Everlaw AI context panel, that the response is based on only a subset of the available content.

Variability of output: Because LLMs are not deterministic systems (ie. there is randomness in their outputs, even with the same input), you should expect some level of variability in the output, even when re-running the same task against the same document.

 📄 Document summaries

Chunked summaries and shorter descriptions

This task generates a narrative summary of the document. In order to preserve sufficient information about the document's content, documents are summarized in chunks, with the size of the chunk scaling with the size of the document. For example, some documents' summary chunks may cover roughly every 5 pages of content, while others may cover every 10 pages.  

If the document is long enough to be summarized over multiple chunks, a shorter description of the document will also be generated. Shorter documents will only have descriptions. 

Document summaries/descriptions can be viewed:

• In the Review Window, via the Everlaw AI tab in the context panel
  • By default, only the description is shown. But you can click on "View summary by section" to expand the view to include the longer, chunked summary, if available.  
• On the results table, via the "Description" column
  • Only the descriptions are shown
• In the "Description" field for a document on the Storybuilder timeline, if there is no human-created description available
  • Only the descriptions are shown. You can learn more about descriptions in Storybuilder here. 

Using summaries in the Review Window

When viewing the chunked summaries in the Review Window, you can click on a page range marker to jump to the first page in the range. This allows you to navigate the document you're viewing by the summary. For longer documents, you can imagine a workflow where you first read the summaries to get the gist of the document, and then hone in on the most pertinent sections utilizing the page range markers. 

You can also take the following actions on summaries in the Review Window:

• Copy to clipboard: copy the contents of the summary or summary chunk to your clipboard
• Copy to note: copy the contents of the summary or summary chunk to the notes field in the Review Window
• Delete: delete the existing summary/description for the document

Generating summaries in batch and exporting summaries

Summaries can be generated in batch from the results table. For the beta, we've imposed a limit of 10,000 documents per batch. To generate a batch summary task:

1. Open the results table, whether associated with a search, binder, upload, etc.
2. Ensure that there are either fewer than 10,000 documents in the table, or fewer than 10,000 documents selected
3. Click "Batch" on the toolbar and select the "Generate descriptions and summaries" option
4. Confirm the task on the dialog that appears, and optionally add the associated "Description" column to your table

After the task is confirmed, the documents will be sent to our queue for processing. Depending on (1) the number of documents submitted and (2) the size of the queue, the batch task can take anywhere from a couple of seconds to many hours to complete. The description column will populate in real-time as individual summary jobs are completed. 

Once descriptions are generated, they can be exported as part of a CSV export. When configuring your CSV export from a results table, simply ensure that the "Description" column is checked. 

Searching based on summaries

You can search for documents based on whether they have AI descriptions/summaries. To search by this status, add the Has Description/Summary search term to your search builder. 

Negate this search to find documents in your project that do not have AI descriptions/summaries.

 

🔖 Topic extraction, with integrated entity and sentiment analysis

This task analyzes the document for topics. Depending on the document's length, there can be anywhere from 1 to over a dozen identified topics.

Note: Sometimes, the names given to the identified topics may make it appear like there are duplicate topics; in such cases, we encourage you to read the summaries of the topics as that will provide information to help you distinguish topics. 

Generating and viewing topics

The topics task is found on the "Overview" tab in the Everlaw AI context panel. To generate topics, simply click the "generate" button in the topics section. 

 

Identified topics will be appear as a list of expandable cards.

Clicking on a collapsed topic card will expand it. 

A topic card has up to 5 components:

• Topic name: The high-level topic. 
• Topic summary: A bullet-point summary for the topic based on the document text.  
• Relevant area: Navigation between document snippets relevant to the topic.  
• Sentiment: The sentiment rating for the topic and an analysis of the rating. The rating will be one of “positive”, “negative”, “harmful and offensive”, or “neutral”. Everlaw will not display any sentiment information if the rating is neutral. 
• Entities: The entities associated with the topic, if any, categorized into organizations and natural persons. Clicking on an entity’s badge will open up a pop-over with additional information about the entity and the option to add the entity name as a hit highlight. 

Navigating topic snippets

For each topic, Everlaw will identify a set of document snippets relevant to the topic. To navigate through these snippets, first open the desired topic, then click the “Show relevant area” component. This will:

1. Assign the topic a highlight color, which will appear on the left edge of the topic card
2. Highlight the related document snippets in that color
3. Replace the “Show relevant area” component with arrows that can be used to navigate between the highlighted document snippets. Only one topic’s snippets can be highlighted and navigated at a time. 

Searching/filtering topics in the Review Window

There are three ways to navigate, search, or filter topics:

• You can use the search option to filter topic cards by keyword. This option only searches/filters by the topic name. 
  • 
• You can jump to and auto-expand a topic card by clicking on it in the Topic menu
  • 
• You can filter topic cards by sentiment rating or entities via the filter option. Multiple selections within the sentiment or entities category are ORed together. Sentiment and entity filters are ANDed together. For example, if a user filters by “negative” and “harmful or offensive”  sentiment and “John Doe” and “Jane Doe”, the filtering logic would be: (sentiment is “negative” OR “harmful or offensive”) AND (entity is “John Doe” OR “Jane Doe”). 
  • 

Additional options in the Review Window

You can copy the information for individual topics to a note or clipboard via the icons in the lower right of the topic card. The ability to copy the information for all topics to a note or clipboard, and to delete the generated topics, can be found via the menu icon for the topic task. 

 

 

Generating topics in batch

You can batch generate topics for up to 2000 documents at a time. To batch generate topics:

1. Open a set of documents. Ensure that there are 2000 documents or fewer selected on the results table.
2. Click Batch in the toolbar and select Topics. A pop-up will appear for you to confirm the action.
3. Optionally, choose to add the Topics column to your results table, if it is not already visible.
4. Submit the batch job.

As topics are generated for documents, topic titles and sentiment ratings are populated in the Topics column. Depending on the number and size of documents, a batch task can take anywhere from a few seconds to many minutes to complete.

You can open a document to review the full topic generation. You can also include the topics column as part of a CSV export. To learn more about exporting, see this article. 

Searching documents by topics

You can search for documents by topic title and sentiment rating. To search by topics:

1. Add the Topics search term to your query
2. If no values are specified in the Title and Sentiment fields, the search will find all documents with generated topics. Negate this search to find documents without any generated topics. 
   • You can search the Title field by keyword or key-phrase, or select a topic from the auto-complete dropdown. 
     • 
   • You can search the Sentiment field by any mixture of sentiment ratings. The sentiment search returns documents that have any topics with that sentiment rating. 
     • 
   • Searching the Title and Sentiment fields together will return documents with one or more topics that match both the topic and sentiment search criteria. 

✄ Custom extractions

Custom extractions allow you to extract out desired pieces of information from documents. For example, if patient records are part of your corpus, you can extract out values from those records – drugs and dosages, medical care providers, etc. – that are important to capture. Custom extractions are useful in cases when keyword or regular expression searches are too ineffectual or cumbersome.

Extractions are based on Fields you configure. A field is a category of information you wish to extract out of the document; there can be multiple extractions per field. For example, if your field is "medications", Everlaw AI will extract out all potential mentions of medications in the document. 

Creating an extraction request

To create an extraction:

1. Open a document in the Review Window
2. Open the Review Assistant context panel on the left sidebar
3. Toggle to the Extractions tab
4. Click Configure and generate extractions to open the configuration view
5. Enter Field, Type, and Description information. Up to 5 fields can be configured for extraction at a time.                                       

• • Field: The name or category of the extraction (ex. contract date, patient name, indemnity language)
  • Type: Choose from:
    • Text: The extractions are expected to be multiple words or sentences in length
    • Number: The extractions are expected to be numbers or references to numerical information
    • DateTime: The extractions are expected to be in datetime format or are references to dates and times
    • Entity: The extractions are expected to be names of entities, such as persons, organizations, or locations. 
  • Description: A concise description of the field

6. Click Generate
7. Wait as the results populate in the extractions table

Extractions are additive: users can configure multiple extractions for a given document, and new results will be added to existing results.

Using extractions

Once extractions are generated, you can:

• Search the table of extractions via the search option on the top right of the table
• Add extractions as hit highlights by clicking the hit highlight icon
• Copy extraction results to your clipboard by clicking the copy to clipboard icon
• Copy extraction results to a note by clicking the note icon

🧐 Document Q&A

This task allows you to ask your own question of a document/create your own task. It can be accessed from the "Ask questions about this document" button in the footer of the AI context panel.

If the document is sufficiently long, the answer will first be returned by document chunk ("detailed" answer), with a final synthesized answer generated at the end ("condensed" answer).

When reviewing the detailed answer, you can click on a page range for a chunk to navigate to the first page in the range. You can also click on the note icon to populate a note field with the content of a generated response, or the clipboard icon to add the generated answer to your clipboard. 

Unlike other review window tasks, the response to this task is private to you. Everlaw will preserve the last 20 questions asked for a given document.