Uploading Native Data to Everlaw

To read more about uploading documents on Everlaw, please refer to the articles in our Uploads section. 

Table of Contents

Everlaw has a cloud-processing system that automatically processes and ingests native files into the platform. Documents processed on Everlaw will go through de-NISTing, deduping, OCR, AV transcription, and language detection, as appropriate. Additionally, Everlaw will generate text, metadata, and PDFs (if requested) for your native data.

A native upload can contain one file, or multiple files. For more information about preparing your files for upload, please see this article

Return to table of contents

Creating a native data upload
 
In order to upload documents to Everlaw, you must have the Upload permission on the database. 
 
To access the Uploads page:
  • Click the project navigation icon in the top right-hand corner of the navigation bar
  • Select “Uploads” from the dropdown menu
  • Select “New Upload” on the left-hand sidebar

go_to_upload_page.gif

Here, you can add the data you want to upload. Please see this article for information on how to prepare your files.

Return to table of contents

Uploading via cloud-based apps: 

At the bottom of the screen, you will see options to upload via popular cloud based storage apps. You can upload directly from the following apps: 

  • Box
  • Dropbox
  • Google Drive 
  • SharePoint
  • OneDrive 
  • Google Vault

Uploading via direct link: 

Direct links are urls that point straight to a file without any password protection. Essentially, if you paste a url into your browser, and your browser starts downloading a file instead of loading a webpage, you have a direct link. You can convert a Google Drive link into a direct link by amending it:

You can also select "direct link" to copy-paste a direct download URL. If you have a URL that, when you enter it in your browser, initiates a download without needing to enter in login credentials, you should be able to paste that URL in the "direct link" field.

Select an app to begin uploading your documents. Please note that if you are uploading documents via OneDrive or SharePoint, you will need to specify whether you are uploading files or folders. This is because OneDrive and SharePoint have separate interfaces for selecting files and folders. Additionally, any top-level folder uploaded via cloud storage source will be compressed and displayed in Everlaw as a container file. 

Once you select an app, you will be asked to log in via a separate dialog box. If you do not see this dialog box, check your pop-up settings. After logging in, you will be able to select your files directly from the app's platform. Once you hit "enter" or "submit" within that app, you will be taken to the next step in the upload process.

For Google Vault users, simply generate exports of your data in Google Vault, and then load the exports into Everlaw directly by choosing Google Vault in the cloud storage options.   

uploads9.gif

When uploading files from Google Drive, note that filename is the only document metadata that will be captured on processing. Fields such as Author, Last Modified Date, and Title will not be captured, even if the Drive file is derived from a file created in Office. During processing, Google Drive files will be given an Office-based extension so that the uploading tool can better understand how to process it. This extension will be included in the filename field upon upload. 

Once you select your file, you will be asked to specify whether you’re uploading native files or processed files. Choose "native."

uploads10.png

Once you’ve made your selection, a wizard will appear where you can specify settings for the upload:

Return to table of contents

Step 1: Dataset details

In this step, you can specify the configuration for your upload. By default, your upload settings will be inherited from your last upload. 

Screen_Shot_2019-06-07_at_4.27.11_PM.png

Name: You are required to give the dataset a unique name. The name (and date of upload) will appear as the name of the associated uploads card on the homepage, which you can rename later if you have upload permissions. 

Deduplication: Upon upload, you can choose to:

  • Deduplicate against all of the existing documents in your database
  • Deduplicate against all of the existing documents by custodian
    • This means that if two duplicates have different custodians, they will both be uploaded. Conversely, if a document has the same custodian as another duplicate document that already exists on the platform, the duplicate file will not be uploaded.
  • Not deduplicate the documents in an upload

To learn more about the definition of duplicates and how upload deduplication handles documents (and families), visit this help article on duplicates. Please note that Google files (e.g., Google Documents, Google Sheets) will not get deduplicated in the same way as other file types, because they undergo their own conversion process within Google. 

Advanced Settings (create PDF's, timezone, OCR language, and email image attachment) can be viewed and edited by clicking the caret icon next to Advanced Settings, which are collapsed by default. 

Create PDFs: By default, Everlaw creates PDF images for all files in an upload, and placeholder images for file types that don’t image well (like spreadsheets). If desired, you can choose to image the file types that don’t image well, or choose to not image any of the files in an upload.

In addition to Excel files, the following file types will also not be imaged by default:

  • LibreOffice Calc
  • Empty files
  • Container files
  • iWorkNumbers
  • QuattroPro

Display Timezone: The timezone you select here will be used as the assumed timezone for metadata fields that lack an explicit timezone value. If you do not select a timezone, then the timezone provided as raw text from the PDF's or emails being uploaded will be displayed. 

OCR Language: This step allows you to specify particular languages for Optical Character Recognition (OCR). OCR will be automatically run on TIFFs and PDF pages with little or no extractable text. By default, OCR language detection is set to Autodetect. Autodetect can extract all Latin-alphabet languages (such as French and German) as well as Chinese, Japanese, and Korean (CJK). If your document has a combination of these aforementioned languages, Autodetect will also be able to OCR them automatically as long as there is only one language per page. Autodetect will not reliably OCR multiple languages within one page.

You can also select a single language to target for OCR. In this mode, OCR will only detect that language and English. There are two scenarios where you would want to select something other than Autodetect:

  1. If all your documents are not in a Latin-alphabet language or CJK (e.g. Russian, Greek)
    • In this case, you must select that language from the dropdown menu in order for OCR to work for that document.
  2. If the quality of your scanned document(s) is low, and you know there is only one language in the document (in addition to English)
    • This will improve the quality of OCR, but only for that language and English. It will prevent the detection of any other languages in that document, so you should be sure that the document only has only one non-English language before selecting that option.

If your upload includes multiple documents, each with different foreign languages, then you’ll want to select Autodetect. However, this means that non-Latin, non-CJK language documents will not get properly OCRed. For example, if one document is entirely in Arabic (non-latin, non-CJK language), and another is in French (Latin language), then only the French document will be properly OCRed. In this situation, you can separate those documents into different uploads so that you can select the appropriate OCR language setting for each, or, after processing, select specific subsets of documents from the results page for reprocessing with a different OCR language.

Page Size: Everlaw will generate PDFs in the selected size for documents that do not have a described size (e.g. emails). Documents with an explicit size (e.g. PDFs, word documents, and images) will remain in their original sizes. Documents exported, printed or produced from Everlaw will respect the size of the pages on the platform. 

Email image attachments: There are three options for deciding whether image attachments should be displayed inline, or treated as separate attachments. If you would like every image in the email to be displayed inline within the PDF, you can choose “Inline all images found in emails." If you would like email image attachments to be extracted as children of the parent email, select “Extract all attached images as children.” Finally, if you choose smart determination, then Everlaw will dynamically determine which images are likely to be attachments, and which ones are (or are intended to be) inlined images (e.g., signature icons). Factors influencing this smart determination include an image's dimensions, overall size, and content ID. 

Passwords: Inaccessible files will not be processed. If any of your files/folders are password-protected, input the password(s) into the password box (one password per line) to enable Everlaw to image and extract text based on your processing options. The native view will not be available for password-protected documents on Everlaw. To overlay a document that can be viewed in native, please follow the instructions in this article.

Return to table of contents

Step 2: Select custodians

uploads13.png

The custodians step allows you to specify what custodian value to associate with the documents you’re uploading. You can specify a default custodian for all documents in an upload and/or set custom custodian values for particular files or folders. If your data belongs to multiple custodians, please read this article to learn how to prepare your data accordingly before uploading.

To set a default custodian, input the custodian name into the “default custodian” box at the top of the table. If your project already has custodians from previous uploads, you can also select one from the dropdown list.

To set custom custodian values for particular files or folders, find the file/folder on the table and input the custodian name into the custodian box on the right. If there is a default custodian, it’ll be overridden for that particular file/folder. Files that have a black caret symbol in the far left can be expanded to display the individual sub-folders/files they contain. Click on the caret icon to expand or collapse. 

Return to table of contents

Step 3: Uploading into partial projects

Aside from uploading the documents into the current project you’re on, you can also add the documents to any partial project you have the Partial Project Document Management permission on. No matter what, documents you upload will automatically be added to all complete projects in the database (i.e., projects that contain all documents in the database). To select or deselect a project, click on the checkbox. You cannot deselect complete projects.

uploads14.png

Once you click ‘upload’, your data will be transferred to our servers. An overlay will appear to show you the status of the transfer. From the overlay, you can add additional documents to the upload by clicking on the “+Add files” button. If you don’t want to add files, close out of the overlay once the transfer has finished and you’ll be able to see your upload's progress.

uploads15.gif

A status card will be added to the native data page corresponding to your upload. A time estimate will appear on this status card to indicate approximately how long it will be until processing is complete. As your upload progresses, you can start reviewing completed docs. You do not need to stay on this page for the upload to continue processing. Once all your files are successfully processed, you will see a document icon with a green checkmark in the status card. Clicking the icon will take you to a results table of your processed documents.  

uploads2.jpg

Native uploads will each be assigned a control number, indicated by a # prefix.

uploads16.png

To learn how to view upload status, rerun, delete, rename, and take other actions, please see the “Managing native uploads” section.

Return to table of contents

Managing native uploads

Uploads will appear as cards in the “Native Data” section of the Uploads page.

Screen_Shot_2019-04-12_at_11.14.35_AM.png

You can take the following actions on an upload from its card:

Screen_Shot_2019-04-12_at_11.23.37_AM.png

1. Rename the upload and/or add a description: Click the upload name and enter a new name. You can add a text description by clicking "Add a description..." Both of these changes will affect the upload across projects in the database. 

2. View uploaded documents: Click the document count (to the right of the document icon) to open the uploaded documents in the results table. You can also access your uploaded documents from the homepage under the Document Sets column. 

3. View upload information and errors: Click "View Report" to see information about deduplicated and deNISTed files, as well as other information related to the upload (e.g., upload errors and issues). See the "Upload Report" section below for more information. 

More options (accessible via the three-dot menu in the top right corner of the upload card):

  • Manage source files: View the progress of your native files' transfer to Everlaw's servers. You can also add additional files to the upload. See section below for more information about adding additional files to your upload. 
  • View configurationView the timezone propagated to your documents and projects the data was uploaded to.
  • Delete: The documents in the upload, including all files generated during processing (e.g., image, text), will be removed from all projects in the database and the database itself. This option is only available to users with the Delete permission

Return to table of contents

View a report of your upload

After uploading your documents, you can view information about it via the upload report. For example, you can see how many documents were deduplicated, as well as how many documents ran into errors during processing. To view a report of your upload, click View Report at the bottom of the upload card. 

Screen_Shot_2019-04-12_at_11.50.53_AM.png 

This will open a visualization of your upload. 

upload_report.png

The pie chart in the middle of the report shows a breakdown of the file types that have been processed and uploaded. You can also see the absolute of number documents for each file type using the list on the left. At the bottom of the report, you can view the number of documents that were OCR’d and imaged, as well as the billable size of the upload. The right panel shows the number of documents that registered errors during processing, broken down by processing stage. Any number in blue is clickable, and will bring you to a results table with the appropriate documents. For more information about troubleshooting native upload errors, please see this article 

At the top, you can see the number and size of documents that were deduplicated or de-NISTed.  To download a report of the documents that were deduplicated upon upload, click "download info." The results of the csv will look like the below: 

csv_download.png

The three column headers upon download are: Original Path, Original Bates, Duplicate Paths. The CSV will display the following information:

  • Native paths for the “original” documents (for each set of duplicates, the single instance of the document that was uploaded to Everlaw)
  • Begin Bates numbers for the original documents
  • Any native paths for the deduplicated documents associated with the original documents (note: there may be multiple native paths)

If there are multiple duplicates for one original path, those duplicates are listed in one line within the Duplicate Paths field. This implies that if one document has multiple duplicates, the total row count in your CSV will be less than the Deduped document count in the report. 

Return to table of contents

Add additional files to your upload

For organizational purposes, it can be helpful to add additional files to an upload after the initial upload is complete. For example, you may want to keep all files from a single custodian in the same upload. If so, you can add additional files to the custodian's upload as they become available to you. To do so, click the three-dot menu in the top right corner of the upload card and choose "Manage source files." 

Screen_Shot_2019-04-12_at_11.36.27_AM.png

From here, click "Add files" in the top right corner. Then, choose where the new files should come from (e.g., local, cloud-based app). 

Screen_Shot_2019-04-12_at_11.39.05_AM.png

You will then be asked to enter any passwords for the new files, and to associate the files with a custodian. The other configuration settings (e.g., deduplication, timezone) will be pulled from the initial upload. 

Return to table of contents

Native data processing settings

  • The orientation of documents is preserved from its native version (e.g., a document that is in landscape orientation will remain that way upon upload)
  • Embedded files: Everlaw will extract all embedded files, including audio and video files, in an Office file (e.g., an Excel file embedded in a PowerPoint) and any file embedded in a PDF.
  • The children of container files are extracted with no limitation on depth. For example, a Word document attached to an email that’s attached to another email that’s in a Zip file that’s in another Zip file will be extracted.
  • Hidden columns in Excel are displayed.
  • Notes are extracted and presented in the PDF/Image view for Word documents and the Native view for spreadsheets.
  • Sometimes, you may try to upload an entire hard drive or a folder with personal files mixed in with system/software files. Some of these files have no user-specific content and can be removed upon processing. This process is called deNIST (removing NIST files). Any files that are on theNIST list will qualify for deNISTing automatically upon upload. Binary files, and virtually all containers, are not part of this list and will not be removed.

Return to table of contents

Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.