Redactor

Redaction is crucial for safeguarding sensitive information across the regulatory industries. Objective already has a product in its suite called "Redact" but it is a desktop-based application used mostly for manual redaction - not for redacting content at scale and in an automated way. This often poses a challenge when dealing with large volumes of documents. The redaction is also limited to only pattern-based detection of sensitive information which often fails to detect some complicated sensitive information like religious beliefs, criminal behavior, etc. This new redaction task will search the content of PDF documents while migrating and redact any words phrases or patterns based on the targets set in the task's configurations.

Note:  Currently the only file type that can be redacted is PDF. You can use the File Format Converter to convert files you wish to redact to PDF.

Features

  • The redaction task is part of the core 3Sixty product capability, which can handle document redaction at scale with minimal human intervention.

  • Supports AI-based sensitive information detection and redaction as a value-add.

  • Supports exemption codes

Configuration

Redaction terms can be added as individual targets. 3Sixty will use each target to determine what in the content needs to be redacted. To use the redaction task, go to the Task tab in your job. Select the task from the drop down and click the plus circle to configure the task. Click done after making any changes to save.

Condition check

It will execute the task when the condition's result is 'true', 't', 'on', '1', or 'yes' (case-insensitive), or run on all conditions if left empty. This condition is evaluated for each document, determining whether the task should be executed based on the specified values.

Example: If I only want to run this task for PDF documents I would use the expression: equals('#{rd.mimetype}',"application/pdf")

Redaction Rules

Terms structure options: Word/phrase or pattern that you want 3Sixty to search for during migration to determine what to redact.

Depending on which option you select you will be prompted to fill in the following

  • Word or Phrase: Enter the word or phrase you want 3Sixty to

  • Pattern: Enter the regex pattern you want 3Sixty to search for and redact

Match case

Check this box to match if the word or pattern has upper or lower case characters that you want to match against.

Whole word

Check this box to only redact if 3Sixty matches the whole word or phrase.

Exemption Code settings

You can apply a code to each target as a way of labeling each redaction

  • Exemption code: Label for the target

  • Description fields: Description of the label

Add Another

Multiple rules can be set as redaction targets. Click the add another button to add more targets.

Redactions Output

  • Working Copy: You can see where the redactions will take place over which words.

  • Finalized: Will actually redact the results

Working Copy Finalized

AI Redaction

Using the AI Completion task users can utilise 3Sixty's new AI feature to redact words and terms. Following are some use cases.

 

You are tasked with detecting PII (personally identifiable information) from a given document text.

You must respond in JSON format.

The response JSON will be used by a redaction system, hence we call it the redaction JSON.

Each PII item must have a "term" and a "code".

The "term" is the exact text to be redacted, and the "code" is either "NAME" for names or "CTRY" for countries.

 

Here are some examples:

Example 1:

Input: Patient John Smith was diagnosed with diabetes at the hospital in New York City.

Expected output:

Copy 
{
"result": [
{
"term": "John Smith",
"code": "NAME"
}
]
}

 

Example 2:

Input: Dr. Jane Doe treated a patient from Canada who had symptoms of COVID-19.

Expected output:

Copy 
{
"result": [
{
"term": "Jane Doe",
"code": "NAME"
},
{
"term": "Canada",
"code": "CTRY"
}
]
}

 

Example 3:

Input: The medical research study was conducted at the University of California, Los Angeles, and involved patients from Australia and the United Kingdom.

Expected output:

Copy 
{
"result": [
{
"term": "Australia",
"code": "CTRY"
},
{
"term": "United Kingdom",
"code": "CTRY"
}
]
}

 

Using the instructions and examples above, create a redaction JSON for the following text.

The text is extracted from a document and can be pretty long.

Copy 
#### INPUT ####
#{field.content}

Properties

Module-Level Toggles

module.ai.enabled

  • Description: Toggles the AI module. Set to true to enable AI functionality or false to disable it.

  • Default Value: false.

module.redaction.enabled

  • Description: Toggles the redaction module. Set to true to enable redaction functionality or false to disable it.

  • Default Value: false.

Redaction-Specific Properties

module.redaction.ocr.confidence-threshold

  • Description: Specifies the minimum confidence level (0-100) for OCR text recognition during redaction. Text with a confidence level below this value will not be considered for redaction.

  • Default Value: 50.

module.redaction.padding.height

  • Description: Determines the percentage increase in height for the redaction box to ensure complete coverage of text. This is useful for ensuring that parts of characters (e.g., 'p' or 'i') are not left visible.

  • Default Value: 50.

module.redaction.background.color

  • Description: Specifies the background color of the redaction box in hexadecimal format (e.g., #RRGGBB). This color is applied to the areas being redacted.

  • Default Value: #000000 (black).

module.redaction.code-text.color

  • Description: Specifies the color of the overlay text in the redaction box in hexadecimal format (e.g., #RRGGBB). This is used for exemption codes or other annotations.

  • Default Value: #FFFFFF (white).

Working Copy Specific Properties

Change the color of the redaction box and exemption codes using the hexadecimal format (e.g., #RRGGBB). These colors are applied to the areas being redacted.

module.redaction.dry-run.background.color

  • Description: Specifies the background color of the redaction box.

  • Default Value: #FFFFFF (white)

module.redaction.dry-run.background.opacity

  • Description: Specifies how transparent the background color of the redaction box is.

  • Default Value: 0.7

module.redaction.dry-run.code-text.color

  • Description: Specifies the color of the exemption code text and the redaction area boarder.

  • Default Value: #0000FF (blue)

Conversion-Specific Properties

module.conversion.ocr.confidence-threshold

  • Description: Specifies the minimum confidence level (0-100) for OCR text recognition during document conversion. Text with a confidence level below this value will not be included in the conversion process.

  • Default Value: 50.

module.conversion.overlay-text.invisible

  • Description: Determines whether the overlay text added during conversion should be invisible or visible. Set to true to make the overlay text invisible, or false to display it. Displaying the overlay text (false) can be useful for debugging or verifying the accuracy of OCR output.

  • Default Value: true.