Document summaries, topic analysis, and Q&A

This article is a deep dive on the summaries, topic analysis, and document Q&A features, which are available in the Everlaw AI beta. 

  • If you're looking for an overview/starter guide for all beta features, see this article.
  • If you're looking for a general overview of the beta program itself (including how to join), or information about our privacy and security standards, please see our Beta FAQ
  • If you have feedback or questions, feel free to email us at beta@everlaw.com. We love hearing from our users.

Table of contents

Overview and background

Accessing the features: Document summaries, topic analysis, and document Q&A can be accessed from the Everlaw AI context panel in the Review Window:

  • The summaries and topic tasks are located in the "Overview" tab.
  • Document Q&A can be accessed from the "Ask questions about this document" button in the footer of the panel

Screenshot 2024-05-02 at 10.16.27 PM.png

In addition, the batch version of the summaries task can be accessed from the results table via the "Batch" icon in the toolbar. 

Screenshot 2024-05-02 at 10.18.18 PM.png

Saving results: Task results are saved per document, per database, with the exception of the document Q&A task, which is saved per user. This means that, for example, a summary generated for a document in one project will also be available for that document in other projects in the same database. 

Deleting results and regenerating: From the Review Window, you can discard a saved result for a task. This means the next time the task is run against the document, a new generation request will be sent to the LLM. The substance of the result for a given task on a given document should rarely change, so you generally do not need to regenerate output, unless there are clear errors in the current result. 

Document text: If the PDF image for a document has extractable text, Everlaw uses that as the basis of analysis. If not, Everlaw falls back on the OCR text file. For batch tasks, Everlaw will always use the OCR text file. Documents that do not have extractable text or an OCR file cannot be analyzed by the Everlaw AI Assistant. 

Document length limits: Everlaw currently limits the length of supportable document text to approximately 300 “pages” (a page being defined as ~500 words). If you run a task against a document that exceeds this threshold, we will let you know, via a banner in the Everlaw AI context panel, that the response is based on only a subset of the available content.

Variability of output: Because LLMs are not deterministic systems (ie. there is randomness in their outputs, even with the same input), you should expect some level of variability in the output, even when re-running the same task against the same document.

 📄 Document summaries

Chunked summaries and shorter descriptions

This task generates a narrative summary of the document. In order to preserve sufficient information about the document's content, documents are summarized in chunks, with the size of the chunk scaling with the size of the document. For example, some documents' summary chunks may cover roughly every 5 pages of content, while others may cover every 10 pages.  

If the document is long enough to be summarized over multiple chunks, a shorter description of the document will also be generated. Shorter documents will only have descriptions. 

Screenshot 2024-05-02 at 10.40.37 PM.png

Document summaries/descriptions can be viewed:

  • In the Review Window, via the Everlaw AI tab in the context panel
    • By default, only the description is shown. But you can click on "View summary by section" to expand the view to include the longer, chunked summary, if available.  
  • On the results table, via the "Description" column
    • Only the descriptions are shown
  • In the "Description" field for a document on the Storybuilder timeline, if there is no human-created description available

Using summaries in the Review Window

When viewing the chunked summaries in the Review Window, you can click on a page range marker to jump to the first page in the range. This allows you to navigate the document you're viewing by the summary. For longer documents, you can imagine a workflow where you first read the summaries to get the gist of the document, and then hone in on the most pertinent sections utilizing the page range markers. 

You can also take the following actions on summaries in the Review Window:

  • Copy to clipboard: copy the contents of the summary or summary chunk to your clipboard
  • Copy to note: copy the contents of the summary or summary chunk to the notes field in the Review Window
  • Delete: delete the existing summary/description for the document

Generating summaries in batch and exporting summaries

Summaries can be generated in batch from the results table. For the beta, we've imposed a limit of 10,000 documents per batch. To generate a batch summary task:

  1. Open the results table, whether associated with a search, binder, upload, etc.
  2. Ensure that there are either fewer than 10,000 documents in the table, or fewer than 10,000 documents selected
  3. Click "Batch" on the toolbar and select the "Generate descriptions and summaries" option
  4. Confirm the task on the dialog that appears, and optionally add the associated "Description" column to your table

After the task is confirmed, the documents will be sent to our queue for processing. Depending on (1) the number of documents submitted and (2) the size of the queue, the batch task can take anywhere from a couple of seconds to many hours to complete. The description column will populate in real-time as individual summary jobs are completed. 

Screenshot 2024-05-02 at 11.08.38 PM.png

Once descriptions are generated, they can be exported as part of a CSV export. When configuring your CSV export from a results table, simply ensure that the "Description" column is checked. 

Screenshot 2024-05-02 at 11.11.57 PM.png

 

🔖 Topic extraction, with integrated entity and sentiment analysis

This task analyzes the document for topics. Depending on the document's length, there can be anywhere from 1 to over a dozen identified topics.

Note: Sometimes, the names given to the identified topics may make it appear like there are duplicate topics; in such cases, we encourage you to read the summaries of the topics as that will provide information to help you distinguish topics. 

Generating and viewing topics

The topics task is found on the "Overview" tab in the Everlaw AI context panel. To generate topics, simply click the "generate" button in the topics section. 

Screenshot 2024-05-02 at 11.22.41 PM.png 

Identified topics will be appear as a list of expandable cards.

Clicking on a collapsed topic card will expand it. 

A topic card has up to 5 components:

  • Topic name: The high-level topic. 
  • Topic summary: A bullet-point summary for the topic based on the document text.  
  • Relevant area: Navigation between document snippets relevant to the topic.  
  • Sentiment: The sentiment rating for the topic and an analysis of the rating. The rating will be one of “positive”, “negative”, “harmful and offensive”, or “neutral”. Everlaw will not display any sentiment information if the rating is neutral. 
  • Entities: The entities associated with the topic, if any, categorized into organizations and natural persons. Clicking on an entity’s badge will open up a pop-over with additional information about the entity and the option to add the entity name as a hit highlight. 

Navigating topic snippets

For each topic, Everlaw will identify a set of document snippets relevant to the topic. To navigate through these snippets, first open the desired topic, then click the “Show relevant area” component. This will:

  1. Assign the topic a highlight color, which will appear on the left edge of the topic card
  2. Highlight the related document snippets in that color
  3. Replace the “Show relevant area” component with arrows that can be used to navigate between the highlighted document snippets. Only one topic’s snippets can be highlighted and navigated at a time. 

Searching/filtering topics

There are three ways to navigate, search, or filter topics:

  • You can use the search option to filter topic cards by keyword. This option only searches/filters by the topic name. 
  • You can jump to and auto-expand a topic card by clicking on it in the Topic menu
  • You can filter topic cards by sentiment rating or entities via the filter option. Multiple selections within the sentiment or entities category are ORed together. Sentiment and entity filters are ANDed together. For example, if a user filters by “negative” and “harmful or offensive”  sentiment and “John Doe” and “Jane Doe”, the filtering logic would be: (sentiment is “negative” OR “harmful or offensive”) AND (entity is “John Doe” OR “Jane Doe”). 

Additional options

You can copy the information for individual topics to a note or clipboard via the icons in the lower right of the topic card. The ability to copy the information for all topics to a note or clipboard, and to delete the generated topics, can be found via the menu icon for the topic task. 

 

 

🧐 Document Q&A

Screenshot 2024-01-02 at 7.15.25 PM.png

This task allows you to ask your own question of a document/create your own task. It can be accessed from the "Ask questions about this document" button in the footer of the AI context panel.

Screenshot 2024-01-02 at 7.20.37 PM.png

If the document is sufficiently long, the answer will first be returned by document chunk ("detailed" answer), with a final synthesized answer generated at the end ("condensed" answer).

Screenshot 2024-01-02 at 7.21.48 PM.png

When reviewing the detailed answer, you can click on a page range for a chunk to navigate to the first page in the range. You can also click on the note icon to populate a note field with the content of a generated response, or the clipboard icon to add the generated answer to your clipboard. 

Unlike other review window tasks, the response to this task is private to you. Everlaw will preserve the last 20 questions asked for a given document. 

Have more questions? Submit a request

0 Comments

Article is closed for comments.