SharePoint REST
Note: UPLOAD SIZE LIMIT
The SharePoint REST API has a hard 2 GB limit on uploads. There is currently no way to circumvent this limit.
Caution: It is recommended that MS Graph be used to Authenticate Microsoft Office products like SharePoint. If users are encountering job fails you should check your SharePoint permissions as the authorised account may not be limiting what 3Sixty can do.
Authentication Connection
There are currently two methods of authentication for the SharePoint REST Connector.

Allowing Custom Apps in SharePoint
Note: CUSTOM APP AUTHENTICATION
This step is only required if you created your O365 tenant after the year 2019. Microsoft disabled custom apps by default, so there is a series of steps needed to turn them on and allow authentication using tokens. This will require the user of Microsoft PowerShell.
-
Install-Module -Name Microsoft.Online.SharePoint.PowerShell
-
$adminUPN="<the full email address of a SharePoint administrator account, example: jdoe@contosotoycompany.onmicrosoft.com>"
-
$orgName="<name of your Office 365 organization, example: contosotoycompany>"
-
$userCredential = Get-Credential -UserName $adminUPN -Message "Type the password."
-
Connect-SPOService -Url https://$orgName-admin.sharepoint.com -Credential $userCredential
-
set-spotenant -DisableCustomAppAuthentication $false
Creating an APP in SharePoint
-
Go to https://**tenant**.sharepoint.com/_layouts/15/appregnew.aspx where tenant is your organization
Note: TENANT PERMISSIONS
If you only wish to register your app for a single tenant site, go to its specific appregnew site. https://**tenant**.sharepoint.com/sites/**tenatSite**/_layouts/15/appregnew.aspx -
Click Generate for Client ID and Client Secret
-
Name the app whatever you wish. 3Sixty will work fine.
-
For app domain use. The host name for wherever 3Sixty is running. If running locally, this can be localhost.
-
Enter the redirect URI.
-
Click Create
-
Make note of your Client ID and Client Secret.
Retrieve Azure Active Directory ID
Setting App Permissions
-
You may still need to set further permissions for your app in order for it to read/write content.
-
In order to do this, go to https://tenant.sharepoint.com/_layouts/15/appinv.aspx
Note: TENANT PERMISSIONS
If you registered your app on a tenant site, go to the following instead: https://**tenant**.sharepoint.com/sites/**tenatSite**/_layouts/15/appinv.aspx -
Set App ID to your apps Client ID and hit "Lookup". The form should populate with your information.
-
For a full list of app permissions, scopes, and rights, see the following reference: Add in Permissions In SharePoint.
-
It is strongly suggested you read through the referenced document to determine the permission level that is acceptable and needed for your organization.
-
All permission requests take the form:
Important: This request cannot contain formatting of any kind or the permissions will not be set
<AppPermissionRequests AllowAppOnlyPolicy="true">
<AppPermissionRequest Scope="http://sharepoint/content/sitecollection/web" Right="FullControl"/>
</AppPermissionRequests>
Tenant sites, ones not linked directly to the team page, so /sites/Test, will require you to perform this process for them as well. In this case, using the SiteCollection scope.
-
Once you have decided on your Scope and Right, input the request into the Permissions Request XML box and click Create. You should see the following.
-
Select Trust It
-
You can check whether your permissions were applied by looking for your app name listed here:
-
https://tenant.sharepoint.com/_layouts/15/appprincipals.aspx?Scope=Web
-

The basic authentication method for SharePoint. The user needs to supply their username and password.
-
The server url: ie https://[tenant].sharepoint.com. IP Address and port will also work. If the instance is secured with an ADFS server, simply click the checkbox at the bottom.
-
Key Records Management Functionalities will not work with this authentication method. For RM, use OAuth
Note: NTLM Authentication - For on premise instances using NTLM, include the full URL, including the subsite you wish to read from (/sites/XXXX). An authentication connection will be required for each site. The username should also include the domain prefix (domain\username)

-
Go to Connectors > Authentication Connections
-
Create a new Connection and selectSharePoint REST OAuthConnector from the drop-down. It will be after the standard SharePoint REST Auth connector. Make sure you select the right one.
OAuth Information
-
SharePoint Online Service URL: Full URL to connect to your SharePoint Online Service. The tenant url is required: ie tenant.sharepoint.com.
-
Azure Active Directory ID: Found at portal.azure.com -> azure active directory -> Properties (id)
-
Access Scope (Optional): Scope is a mechanism in OAuth 2.0 to limit an application's access to a user's account
-
Tenant Site Path: Due to how 3Sixty performs connection checks, this field is required if your connection is only registered to a single tenant site. Otherwise, the check will give a false negative.
-
How often should 3Sixty refresh the authentication token?: Allows the user to set how often the authentication token will be refreshed. For larger runs where transactions may take longer, this will prevent authentication from expiring while transactions are still in progress.
-
Client ID: Provided to you during your App registration (more info)
-
Client Secret: Provided to you during your App registration (more info)
-
Access Token: Click Authenticate before saving to populate the Access token
-
Expires: The date the token expires
Proxy Information Tab: This tab is for if you're connecting through a proxy, and is optional.
-
ProxyURL: The URL of the proxy server
-
ProxyPort: The port of the proxy server
-
ProxyUsername: (Optional) The username to authenticate to the proxy server
-
ProxyPassword: (Optional) The password to authenticate to the proxy server
Connecting in 3Sixty
-
Input the required information and hit "Authenticate".
-
For the Azure AD Connector, the tokens should automatically populate. Click Save.
-
-
You should see the "trust it" page again.
-
Click Trust It. You should return to your Auth Connection page, with all the appropriate fields populated.
-
Click Save.

Instance configuration
-
Name: Unique Name for the Discovery Connection to identify it in the UI.
-
Authentication Connection: The Authentication connection to the SharePoint Server you want to discover
-
Ignore Types: Comma delimited list of types to ignore. Note that you can have regex as well. So to ignore all types with "workflow" in the name, you would enter(.)workflow(.) into the ignore types textbox.
-
Site path: Path to the Site you want to discover(i.e. /mysite/mysubsite). Enter a single forward slash / for root site (this will discover your entire site). Required to run discovery on tenant sites (ie, [sharepointURL]/sites/mysite)
-
NTLM: Leave blank. Include this on the URL in the authentication connection.
-
Once configured click save and then on the Run link to execute the discovery. This could take quite a while depending on what site you decided to start at. For instance, a system with a thousand sites and 10 thousand libraries will take a while.
Integration Connection
The connection can operate in both repo and output modes. In repo mode, it will retrieve list items and all of their relevant metadata from a list or library on the specified site.
In output mode, the connection will write content and assign the mapped content type (from type mappings), or simply leave the new list item as a Document
Note: FIELD MAPPINGS
A note on SharePoint REST Mappings. The only field type that needs the target type set for a mapping are Date fields. Numbers and text are parsed automatically from Strings.
Important: This connector doesn't support Manage Metadata Column Types
Configuration
-
Connection Name: This is a unique name given to the connector instance upon creation.
-
Description: A description of the connector to help identify it better.
-
Authentication Connection: One of the SharePoint REST Authentication Connections
Job Configuration
Mapping
If you want to map the document version comments, please add below mapping (mapping is shown from SharePoint REST to CMIS connector)
Repository
-
List of Sites to Crawl: The sites from which this connector is collecting data. Use 'root' for the root site.
-
Libraries to Crawl: Which is the Shared Documents' folder in your site. It will crawl this lists in sequence. If left blank it will default to "Documents". List names do not require their parent sites (ex. 'Documents' not 'sites/Test/Documents')
-
Crawl all Libraries: Search all libraries. Checking this option will ignore the libraries listed.
-
Crawl subsites: Search the subsites of your library list.
-
Get Versions : Will gather document versions and binaries (if include binaries is selected). This option will reduce processing speed of documents.
-
Process Folders: As the name suggests, will gather folder metadata for processing/output.
-
Get Permissions: Enables the gathering of security information from the site. If this option is selected, the selected level of security will be applied to each document.
-
Parser Date Time Format: The date/time format of where the SharePoint server exists. Use country codes such as AU or US, or date format such as MM/dd/yyyy hh:mm a
Important: DATE/TIME SETTINGS
Check the settings for any existing jobs to be sure the date/time is set to US.
To confirm the location try running a test by running a SP to BFS job and check to see if the dates are accurate. Dates with day of month over 12 like July 25th would create an issue if converted from 07/25/2022 to 25/07/2022
Output
Important: SharePoint REST connector doesn't check for common illegal characters in file names before attempting to upload a file. The list of characters to avoid is % * : < > ? / |
-
Site Name : The name of the sub site or tenant site where the data will be placed.
-
List Title : The name of the library where the data will be placed.
-
Output Folder Path : The output folder inside the Library. The folder will be created if it does not exist.
Content Service Connection
Connection Configuration
-
Site Name : The site from which this connector is collecting data.
-
Site Url: [tenant].simflofy.com/sites/test, Value: sites/Test
-
Site URL: [tenant].sharepoint.com/MySite, Value: MySite
-
NTLM: Leave blank. Include this on the URL in the authentication connection.
-
-
List Title :The name of the library the connection will interact with. Connections can only support one library. The default value, if left blank, will be the Documents library, which will write to the Shared Documents' folder of your site.
If you're going to be testing REST endpoints, you will need a REST client to make the DELETE and POST calls.
Refer to this tutorial to set up Postman.
cUrl calls will also work, but the Postman interface makes it all much easier.
API Keys
SharePoint REST Connector: Read=true: Write=true: MIP=false
Repo (Read) Specs
Key |
Description |
Data Type |
---|---|---|
siteName | Site Name (or path, if part of a site group, ie sites/MySite). | String |
listName |
List Title |
String |
getVersions |
Get Versions |
Boolean |
getPermissions |
Get Permissions |
Boolean |
dateTimeParser |
Parser Date & Time format |
String |
Output (Write) Specs
Key |
Description |
Data Type |
---|---|---|
siteNameOut | Site Name (or path, if part of a site group, ie sites/MySite). | String |
listNameOut |
List Title |
String |
outputfolderpath |
Output Folder Path |
String |
chunkRetries |
How many times should 3Sixty attempt to start a chunked upload. |
Integer |
chunkRetryWait |
How long to wait between checks for the target file when performing a chunked upload (milliseconds). |
Integer |
Related Articles