Permissions for RAG and AI [with MS]

Permissions are a necessary consideration for any use case that involves file/data ingestion. For use cases like RAG, your AI application will need to have an authentication and authorization strategy for data ingested from third-party providers like Google Drive, Dropbox, Salesforce, etc, so users can only get answers with data they are permitted to.

As you build for enterprise customers, it’s very common for their admins or IT departments to authorize data access/syncs for their entire company. You certainly wouldn’t want an employee of one of your enterprise customers to be able to query for performance review or payroll data unless they had the proper permissions!

Paragon’s Managed Sync product solves for this use case with the Permissions API.

• We’ll start with context on native 3rd-party permissions

• We’ll then walk through how the Permissions API simplifies permissions for 3rd-party data

Native 3rd-Party Permissions

Native Google Drive, Box, Salesforce, Confluence, etc. permissions need to be taken into account if your application is using that data and returning it to end-users.

Normally, if you’re ingesting and storing 3rd-party data like text from Google Drive, you would also want to store the Google Drive permissions data in an access control list (ACL) or graph database to track what users have access to what data.

As you can see, storing and managing permissions can get messy, especially when storing native permissions for multiple 3rd-parties. Not to mention you also need to make sure you have the latest permissions via scheduled syncs or webhooks.

The Permissions API manages the native permissions storage and refreshes for you, allowing you to just call the API and check if users have access to data.

How the Permissions API works

With the Permissions API under Managed Sync, you don’t need to worry about pulling down Google Drive, Salesforce, or any other 3rd-party permissions. When you’ve synced your users’ data with Managed Sync, the permissions for all your users’ 3rd-party data is also synced and stored in Paragon’s managed infrastructure.

The Permissions API provides an interface to easily check access to synced data. There are a few ways to use the API to enforce native 3rd-party permissions.

1. Checking access post-query

curl --request POST \\
  --url <https://managed-sync.useparagon.com/permissions/check> \\
  --header 'Authorization: Bearer <token>' \\
  --data '{
  "object": {},
  "user": {},
  "role": "<string>"
}'

If you’re querying data from a vector database for RAG, you can store the 3rd-party IDs in the record metadata. If you have a relational database, you can similarly keep the 3rd-party ID in a column. From there, you can use the Permissions API endpoint above to check if a user has access to each object returned from a query.

2. Filtering access in-query

curl --request POST \\
  --url https://managed-sync.useparagon.com/permissions/list-objects \\
  --header 'Content-Type: application/json' \\
  --data '{
  "user": {},
  "role": "<string>"
}'

A second method of enforcing permissions is to first call the Permissions API to get a list of objects a user has access to. From there, you can use a filter statement in your query to the database to apply permissions to data returned to your end-users.

As you can see, Permissions API simplifies respecting native 3rd-party permissions to just one API call, while providing a few different methods of implementation for developer flexibility. Check out the full Permissions API docs for more details.

Wrapping Up

Build production-ready data systems with permissions seamlessly with Paragon’s Permissions API. Paragon has helped AI companies sync data and handle permissions. Let us help help you build out a robust permissions system for your application as well. Reach out and book a demo with our team.