SharePoint REST

Note:  UPLOAD SIZE LIMIT
The SharePoint REST API has a hard 2 GB limit on uploads. There is currently no way to circumvent this limit.

Caution:  It is recommended that MS Graph be used to Authenticate Microsoft Office products like SharePoint. If users are encountering job fails you should check your SharePoint permissions as the authorised account may not be limiting what 3Sixty can do.

Authentication Connection

This connector require an Authentication Connection to function. Authentication connectors are used to authenticate repository/output connections that need certain authentication fields like access tokens or refresh tokens.

  1. Select Authentication from the navigation menu or the dashboard.

  2. Click the New Authentication button

  3. Enter a unique name for the connection and give it a description

  4. Select the connection type from the drop-down and click save.

  5. Fill in the configuration settings on the edit connection page and click save.

Authentication Connections Overview

There are currently two methods of authentication for the SharePoint. SharePoint REST Authentication Connection and SharePoint REST Azure AD OAuth Connection

To create a SharePoint Authentication Connection in 3Sixty go to Connectors > Authentication Connections. Click the New Connection button and select your desired authentication method SharePoint REST or SharePoint REST OAuth Connector from the drop-down. Make sure you select the right one.

Note:  NTLM Authentication - For on premise instances using NTLM, include the full URL, including the subsite you wish to read from (/sites/XXXX). An authentication connection will be required for each site. The username should also include the domain prefix (domain\username)

SharePoint REST

The basic authentication method for SharePoint. The user needs to supply their username and password.

The server url: ie https://[tenant].sharepoint.com. IP Address and port will also work. If the instance is secured with an ADFS server, simply click the checkbox at the bottom.

Key Records Management Functionalities will not work with this authentication method. For RM, use OAuth

SharePoint OAuth

Using the Azure AD Authentication you will need to set up an app in 3Sixty by following the steps below.

Allowing Custom Apps in SharePoint

Note:  CUSTOM APP AUTHENTICATION
This step is only required if you created your O365 tenant after the year 2019. Microsoft disabled custom apps by default, so there is a series of steps needed to turn them on and allow authentication using tokens. This will require the user of Microsoft PowerShell.

  1. Install-Module -Name Microsoft.Online.SharePoint.PowerShell

  2. $adminUPN="<the full email address of a SharePoint administrator account, example: jdoe@contosotoycompany.onmicrosoft.com>"

  3. $orgName="<name of your Office 365 organization, example: contosotoycompany>"

  4. $userCredential = Get-Credential -UserName $adminUPN -Message "Type the password."

  5. Connect-SPOService -Url https://$orgName-admin.sharepoint.com -Credential $userCredential

  6. set-spotenant -DisableCustomAppAuthentication $false

Creating an APP in SharePoint

  • Go to https://**tenant**.sharepoint.com/_layouts/15/appregnew.aspx where tenant is your organization

    Note:  TENANT PERMISSIONS
    If you only wish to register your app for a single tenant site, go to its specific appregnew site. https://**tenant**.sharepoint.com/sites/**tenatSite**/_layouts/15/appregnew.aspx

  • Click Generate for Client ID and Client Secret

  • Name the app whatever you wish. 3Sixty will work fine.

  • For app domain use. The host name for wherever 3Sixty is running. If running locally, this can be localhost.

  • Enter the redirect URI.

  • Click Create

  • Make note of your Client ID and Client Secret.

Retrieve Azure Active Directory ID

  • Go to portal.azure.com-> azure active directory -> Properties (id)

  • Retrieve the Directory ID.

Setting App Permissions

  • You may still need to set further permissions for your app in order for it to read/write content.

  • In order to do this, go to https://tenant.sharepoint.com/_layouts/15/appinv.aspx

    Note:  TENANT PERMISSIONS
    If you registered your app on a tenant site, go to the following instead: https://**tenant**.sharepoint.com/sites/**tenatSite**/_layouts/15/appinv.aspx

  • Set App ID to your apps Client ID and hit "Lookup". The form should populate with your information.

  • For a full list of app permissions, scopes, and rights, see the following reference: Add in Permissions In SharePoint.

  • It is strongly suggested you read through the referenced document to determine the permission level that is acceptable and needed for your organization.

  • All permission requests take the form:

Important:  This request cannot contain formatting of any kind or the permissions will not be set

<AppPermissionRequests AllowAppOnlyPolicy="true">
<AppPermissionRequest Scope="http://sharepoint/content/sitecollection/web" Right="FullControl"/>
</AppPermissionRequests>

Tenant sites, ones not linked directly to the team page, so /sites/Test, will require you to perform this process for them as well. In this case, using the SiteCollection scope.

  • Once you have decided on your Scope and Right, input the request into the Permissions Request XML box and click Create. You should see the following.

  • Select Trust It

  • You can check whether your permissions were applied by looking for your app name listed here:

    • https://tenant.sharepoint.com/_layouts/15/appprincipals.aspx?Scope=Web

SharePoint REST Azure AD OAuth Connection Configuration

  • SharePoint Online Service URL: Full URL to connect to your SharePoint Online Service. The tenant url is required: ie tenant.sharepoint.com.

  • Azure Active Directory ID: Found at portal.azure.com -> azure active directory -> Properties (id)

  • Access Scope (Optional): Scope is a mechanism in OAuth 2.0 to limit an application's access to a user's account

  • Tenant Site Path: Due to how 3Sixty performs connection checks, this field is required if your connection is only registered to a single tenant site. Otherwise, the check will give a false negative.

  • How often should 3Sixty refresh the authentication token?: Allows the user to set how often the authentication token will be refreshed. For larger runs where transactions may take longer, this will prevent authentication from expiring while transactions are still in progress.

  • Client ID: Provided to you during your App registration

  • Client Secret: Provided to you during your App registration

  • Access Token: Click Authenticate before saving to populate the Access token

  • Expires: The date the token expires

 

Proxy Information

This tab is for if you're connecting through a proxy, and is optional.

  • ProxyURL: The URL of the proxy server

  • ProxyPort: The port of the proxy server

  • ProxyUsername: (Optional) The username to authenticate to the proxy server

  • ProxyPassword: (Optional) The password to authenticate to the proxy server

Connecting in 3Sixty

  • Input the required information and hit "Authenticate".

    • For the Azure AD Connector, the tokens should automatically populate. Click Save.

  • You should see the "trust it" page again.

  • Click Trust It. You should return to your Auth Connection page, with all the appropriate fields populated.

  • Click Save.

 

Discovery Connection

Discovery makes mapping to and from your sources easier for data migration and ensures the data gets connected to the right fields. Discovery requires an Authentication Connection

  1. Select Schema Instance from the Navigation Menu or the Dashboard under Discovery

  2. Select the new connection button

  3. Enter a name for your Discovery Connection

  4. Select your connection type from the drop down

  5. Click save on the Create Connection form

  6. In the edit connection page fill out any addition fields and click save

Guide to Discovery Connections

Instance configuration

  • Name: Unique Name for the Discovery Connection to identify it in the UI.

  • Authentication Connection: The Authentication connection to the SharePoint Server you want to discover

  • Ignore Types: Comma delimited list of types to ignore. Note that you can have regex as well. So to ignore all types with "workflow" in the name, you would enter(.)workflow(.) into the ignore types textbox.

  • Site path: Path to the Site you want to discover(i.e. /mysite/mysubsite). Enter a single forward slash / for root site (this will discover your entire site). Required to run discovery on tenant sites (ie, [sharepointURL]/sites/mysite)

    • NTLM: Leave blank. Include this on the URL in the authentication connection.

Once configured click save and then on the Run link to execute the discovery. This could take quite a while depending on what site you decided to start at. For instance, a system with a thousand sites and 10 thousand libraries will take a while.

Integration Connection

Every job requires an integration connection for both the source repository connection and the output connection. Also known as input an output connections. Their job is to query or crawl remote systems for files, folders, metadata, versions, and renditions. . In repo mode, it will retrieve list items and all of their relevant metadata from a list or library on the specified site. In output mode, the connection will write content and assign the mapped content type (from type mappings), or simply leave the new list item as a Document. Click here for more information on setting up an integration connection.

  1. Select integration and click on the new connection button.

  2. Enter the name and description of your connection.

  3. Select the connection type from the drop down list.

  4. Click Save on the Create Connection form.

  5. Click Save on the Edit Connection page.

There are no fields to configure in an integration connection.

Guide toIntegration connections

Note:  FIELD MAPPINGS
A note on SharePoint REST Mappings. The only field type that needs the target type set for a mapping are Date fields. Numbers and text are parsed automatically from Strings.

Important:  This connector doesn't support Manage Metadata Column Types

Configuration

  • Connection Name: This is a unique name given to the connector instance upon creation.

  • Description: A description of the connector to help identify it better.

  • Authentication Connection: One of the SharePoint REST Authentication Connections

Job Configuration

A 3Sixty Job is the process of moving or syncing content (including versions, ACL's, metadata) from one CMS (content management system) to another. Add tasks to your job to have better control over how your data gets migrated. Click here for details on how to set up an integration job.

Guide to Integration jobs

  1. Select List Jobs under integration on the navigation menu or the dashboard

  2. Click the create job button

  3. In the New Job form

    1. Name the job

    2. Select Simple Migration from the job type drop down

    3. Select the Repository Connection

    4. Select the Output connection

    5. Select your Content Service Connection

      (Only required if you will be using Federation)

    6. Click Save to open the Edit Job page

    7. Fill in the configurations for the Repo and Output Configuration tabs

  4. Click save to save your new integration job

  1. Select Run and Monitor Jobs under Integration in the navigation menu

  2. Click the play button next to the job you want to run

  3. Click the refresh button to view your completed job status

    (Larger jobs will take longer to run)

Repository

Configuration settings for 3Sixty v4.5

  • List of Sites to Crawl: The sites from which this connector is collecting data. Use 'root' for the root site.

  • Libraries to Crawl: Which is the Shared Documents' folder in your site. It will crawl this lists in sequence. If left blank it will default to "Documents". List names do not require their parent sites (ex. 'Documents' not 'sites/Test/Documents')

  • Crawl all Libraries: Search all libraries. Checking this option will ignore the libraries listed.

  • Crawl subsites: Search the subsites of your library list.

  • Get Versions : Will gather document versions and binaries (if include binaries is selected). This option will reduce processing speed of documents.

  • Process Folders: As the name suggests, will gather folder metadata for processing/output.

  • Get Permissions: Enables the gathering of security information from the site. If this option is selected, the selected level of security will be applied to each document.

  • Parser Date Time Format: The date/time format of where the SharePoint server exists. Use country codes such as AU or US, or date format such as MM/dd/yyyy hh:mm a

Important:  DATE/TIME SETTINGS
Check the settings for any existing jobs to be sure the date/time is set to US.
To confirm the location try running a test by running a SP to BFS job and check to see if the dates are accurate. Dates with day of month over 12 like July 25th would create an issue if converted from 07/25/2022 to 25/07/2022

Output

Important:  SharePoint REST connector doesn't check for common illegal characters in file names before attempting to upload a file. The list of characters to avoid is % * : < > ? / |

  • Site Name: The name of the sub site or tenant site where the data will be placed.

  • List Title: The name of the library where the data will be placed.

  • Output Folder Path: The output folder inside the Library. The folder will be created if it does not exist.

Mapping

3Sixty Mapping gives you the ability to map your content types and metadata from one system to another.

There are two places in 3Sixty where you can set up mappings:

  • At the top of the mapping tab you can select from saved mappings

  • Creating a unique mapping based on your connectors.

Guide to 3Sixty Mapping

Tasks

Job tasks also known as processors provide a processing pipeline for documents and metadata. Post processors can also be added as a task to run after the files are processed.

Click here for a list of all the tasks you can add to your integration job.

  1. Click on the task tab in your integration job.

  2. Find the task you want to create in the drop down. Tasks are grouped by type. You can also search for the task by name.

  3. Select the create task button

  4. Fill out the configuration fields. Some tasks do not require configuration.

  5. Click Done to save the task to your job.

Content Service Connection

Connection Configuration

  • Site Name : The site from which this connector is collecting data.

    • Site Url: [tenant].simflofy.com/sites/test, Value: sites/Test

    • Site URL: [tenant].sharepoint.com/MySite, Value: MySite

    • NTLM: Leave blank. Include this on the URL in the authentication connection.

  • List Title: The name of the library the connection will interact with. Connections can only support one library. The default value, if left blank, will be the Documents library, which will write to the Shared Documents' folder of your site.

If you're going to be testing REST endpoints, you will need a REST client to make the DELETE and POST calls.

Refer to this tutorial to set up Postman.

cUrl calls will also work, but the Postman interface makes it all much easier.

 

If you want to map the document version comments, please add below mapping (mapping is shown from SharePoint REST to CMIS connector)

Source

Mapping Type

Target

Target Type

Document.CheckinComment

Field Mapping

cmis:checkincomment

String

 

API Keys

SharePoint REST Connector: Read=true: Write=true: MIP=false

Repo (Read) Specs

Key

Description

Data Type
siteName Site Name (or path, if part of a site group, ie sites/MySite). String

listName

List Title

String

getVersions

Get Versions

Boolean

getPermissions

Get Permissions

Boolean

dateTimeParser

Parser Date & Time format

String

Output (Write) Specs

Key

Description

Data Type
siteNameOut Site Name (or path, if part of a site group, ie sites/MySite). String

listNameOut

List Title

String

outputfolderpath

Output Folder Path

String

chunkRetries

How many times should 3Sixty attempt to start a chunked upload.

Integer

chunkRetryWait

How long to wait between checks for the target file when performing a chunked upload (milliseconds).

Integer

Related Articles

SharePoint API Endpoints

Managing SharePoint Permissions with Content Services

Retrieving SharePoint ID's Using Content Service