Overlaying Data

Once your data is uploaded into Everlaw, it's possible that you will need to modify it. For example, you might want to fill in missing data, implement claw back requests, or bulk update metadata values.

Overlaying data allows you to modify existing documents in your database by replacing or adding image files, text files, native files, and/or metadata. These modifications can be done while maintaining any work product already applied to documents (codes, notes, ratings, etc.). Highlights and redactions are also maintained, although their positioning could change if documents are overlaid with a different size or shape of image.

Important: Always use caution when performing overlays. They cannot be undone or deleted.

Table of Contents

• Required permissions
• Prepare your data for an overlay
  • Prepare to overlay metadata only
    • Editable metadata
  • Prepare to replace image, text, and/or native files 
• Perform an overlay
  
  • Step 1: Start a new upload
  • Step 2: New Upload
  • Step 3: Load file
  • Step 4: Identity Fields
  • Step 5: Found existing documents
  • Step 6: Metadata type and format 
  • Step 7: Combine Metadata
  • Step 8: Metadata mapping
  • Step 9: Delete metadata
  • Step 10: Additional Options
  • Step 11: Summary
  • Step 12: Images, natives, text
  • Completed upload
• Results of overlays

Required permissions

To perform an overlay you must have Upload or Database Administrator permissions. To learn more about upload permissions, visit Database Administration and Permissions.

Prepare your data for an overlay

Overlays are done through the processed data uploader. This section describes how to prepare and structure your data so it's ready to be uploaded.

Prepare to overlay metadata only

Your upload folder only needs to contain a load file, which can be a CSV or DAT file. The load file must have one field, or column, containing the BeginBates or Control Number. This field is used to match against existing documents in the database. It should also have a field (column) for any other metadata information you want to update, including metadata values you'd like to delete.

If you have been provided a load file containing the correct metadata information, please skip ahead to Performing the overlay.

If you need to generate the load file to use for the overlay, you can export one from Everlaw and then change and add the metadata as necessary. To do this: 

1. Generate a load file on Everlaw by exporting a CSV from a results table displaying the documents you will overlay. Be sure to include the Bates/Control number and all the metadata fields you want to modify. Do not export any categories or codes.

   
2. Open this CSV in a spreadsheet application to change or add values and fields. You can: 
   • Edit the values for a specific field 
   • Add new columns (fields) and fill in the values for your documents. If these new column headings don't already exist as metadata fields in your Everlaw database, they will create new metadata fields when the overlay is complete. 
   • Delete values from fields. Overlaying empty values will remove the metadata information on Everlaw.
3. Place the updated CSV in a new folder. In typical uploads this folder is titled "Data," but any title you choose is fine; the folder title will not affect the upload process. 

Editable metadata

The values for editable metadata fields do not change when updated by an overlay. To make sure that all your intended fields are updated, you can check the Project Settings page to make sure none of the fields you overlay are editable.

Prepare to replace image, text, and/or native files 

To replace image, text, and/or native files: 

• You must have one sub-folder for each file type you are replacing or adding.
  • If these subfolders have not already been provided to you, you will need to make them on your computer and add the appropriate files into each. The ultimate folder structure might look like one of these, depending on which files are included in your overlay:
    

• Image and text file names must exactly match their Bates or control number on Everlaw.  
• You must have a load file containing each document's unique identifier (Begin Bates or Control number). These values must exactly match their values on Everlaw.
• Any metadata information that you want to add or replace should be contained within the load file (see above section for preparing for a metadata overlay) 
• If you are overlaying native files, the load file must contain a Native Path field

For more information on formatting your folders, visit the standard format for processed data article.

Perform an overlay

Overlaying data follows similar steps to uploading processed data with a load file. If you are familiar with this process, most steps of overlaying data will also be familiar. This section will walk you through your overlay upload, with details specific to the overlay process.

 

Step 1: Start a new upload

1. From the homepage, go to Data Transfer > Uploads
   
2. Select Processed.
3. For Type of processed upload, choose Document sets with load file.
   
4. Click Start upload

Step 2: New Upload

1. To import the folder containing your overlay, drag and drop it on the page.
   This opens the New processed upload details dialog.
2. In Upload name, enter a name for your upload.
3. in Authorized by, select your name.
4. In Visibility, select whether or not you'd like this upload visible from the homepage.
5. Click Create upload.

Step 3: Load file

1. If there are multiple folders, use the caret icon to expand the folder containing your load file. In most cases, the relevant folder will be titled "Data."
   
   
2. Select your load file from the folder.
3. Select Upload load file.

 

Step 4: Identity Fields

1.  In the Confirm fields column, the uploader should automatically recognize the Begin Bates of your documents. You can read more in the processed uploads article.
2. You should also see the metadata fields you're adding or changing, along with some randomly sampled values from your load file. 
   1. Note: if your load file has empty metadata fields, the uploader tool will automatically ignore them during ingestion. If you’re overlaying an empty column in order to delete the values of that metadata field, then you must check the box in the Include column. 
3. Once you have confirmed that the information on the Identity Fields page is correct, select Next.
4. If you are overlaying the All Custodians field, you will see a box asking how you'd like to proceed. Make your selection and click Continue. 
   •  Note: if you choose to continue with the current overlay, the All Custodians field will no longer update when new native documents are uploaded to this database. We do not recommend overlaying to the All Custodians field. Please reach out to support@everlaw.com for more information about overlaying Custodian and All Custodians metadata.

Step 5: Found existing documents

1. One key check is to verify that the Found existing documents step appears after confirming the identity fields. This confirms that the Begin Bates of documents in your upload match those of documents already in your database and are not new documents.

   

   If this step does not appear, then your load file was misconfigured and Everlaw does not recognize the documents as already existing on Everlaw. 

   You can select any combination of text, image, native, or metadata to overlay, depending on what is present in the upload folder you selected at the beginning of the upload.

   •  The native option will only be available if a Native Path field was identified in the previous identity step
   • If you are only performing a metadata overlay and there are no text, native, or image files, then you should only select Metadata
2. Select Next

Step 6: Metadata type and format 

1. Confirm that Everlaw has detected the correct data type for each of your metadata fields.
2. Adjust the type if any field type was not automatically detected by Everlaw. Read more in the metadata type and format section of our uploading processed data article.
3. When all the metadata fields have been correctly typed, select Confirm types.

Step 7: Combine Metadata

1.  Confirm the combination of date and time metadata fields. If you do not want to combine these, choose to split them. Read more in the combine metadata section of our uploading processed data article.
   • Note: there might not be any recommended combinations, as this step is dependent on the datetime fields in your load file

Step 8: Metadata mapping

1. You can read details about metadata mapping in our processed uploads section. If you are overlaying metadata, please pay attention to the following during the metadata mapping step:
   • If you are updating metadata: make sure the loadfile header is mapped to the intended field. For example, if you are updating values in the "Date Sent" field, make sure that this field is accurately mapped to "Date Sent"
   • If you are adding a new metadata field, make sure you are not mapping it to a field that already has metadata in it. A new custom field will have a yellow band next to its name when you select it. An existing field will have a blue band next to its name.  If you map this new field to an existing field, the values in the existing field will get replaced.
   • If you are deleting metadata: the field that you are removing values from will not be present in the Metadata mapping step. You will confirm the deletion in the next step.
2. Select Confirm mappings.
3. If you are updating or changing metadata values: you may be taken to a page to confirm the metadata mapping for any fields that are being updated. If all the fields look good, select Confirm configuration. Otherwise, select Reconfigure mappings.

Step 9: If you are deleting metadata: Metadata deletion

1. Select Select a field in the Field to delete column 
2. Select the field from which you are deleting values 
3. Select Confirm deletion
4. You will see this warning. "Mapping an empty column from your load file will delete all the existing values of the fields you selected. This action cannot be undone."
   • Note: This will only delete the metadata values of the Begin Bates/Control Numbers specified in your load file. It will not delete the values of the other documents in your project.
5. Select Confirm deletion again.

Step 10: Additional Options

1. If you'd like to apply work product to these documents as part of the overlay, you can add it here. See the additional options section from uploading processed data article to learn more.
   • Any applied codes will be added to existing coding, with the exception of mutually exclusive categories where the codes assigned at upload will take precedence
   • The custom values applied to existing metadata fields during upload to take precedence over any existing values for those fields
2. Select Submit work product.

Step 11: Summary

1. Review the summary of your overlay.
2. If everything looks good, select Confirm configuration.  If there is anything you'd like to change, select Reconfigure work product to move backward in the process.
3. If you are only overlaying metadata, the upload will begin. If your overlay includes image files, native files, or text files, you will identify them in the next step.

Step 12: Images, natives, text

If your upload includes images, natives, and/or text files, you will select their respective folders here.

1. If you are overlaying image files, select the folder containing your images
2. If you are overlaying native files, click Select folders under Natives and then select the folder containing your natives
3. If you are overlaying text files, click Select folders under Text and then select the folder containing your text files. If you do not have text files, you can select the images folder again. It won't affect your upload — these images won't be uploaded twice.
4. Click Verify files

Completed upload

Once the upload is submitted and processed, the upload summary card provides information about the overlay under the Changed documents section, including success or errors and the number of documents affected.

Results of overlays

Here's what you can expect to happen after an overlay:

• Image and text files: The new files will replace the old ones and the old ones will be deleted. 
• Native files: The new native files will replace the old ones and the old ones will be deleted. However, metadata will not automatically be modified even if the new native files have different metadata than the old files. You must either (1) include the metadata overlay or (2) reprocess the new native files to have the project metadata reflect the new native files' metadata. 
• Metadata: New metadata fields and values will be added. If the overlay contains fields that map to existing fields, the existing values for those fields will be replaced by the new values found in the overlay file. If you overlaid empty values to an existing value, the values in those fields will be deleted.

Note: Partial project selection only affects newly uploaded documents, not overlays.

To learn more, visit Applying Custom Metadata Values and Work Product During Upload.

The overlay workflow cannot not be used to import or overlay review product. To import review product, use the special purpose review product import tool.