Introducing Box Skills

Quickstart

Choose a language to get started:

What is a Box Skill?

A Box Skill is a type of application that performs custom processing for files uploaded to Box.

Often, Box Skills leverage third-party AI/ML services to automatically extract information from files upon upload to Box. For example, a Box Skill could automatically label objects in images using a computer vision service.

The framework allows a Box Skill to receive information about a file as it’s uploaded to Box, retrieve the file for processing, and write to the file’s metadata in Box. Metadata applied by a Box Skill can be displayed to end users via the Box application and can drive other capabilities of the Box platform, such as search, workflow, and policies.

Box Skills are enabled in a Box environment by the primary Box admin. The admin can define which folder(s) a Skill can process. The admin can authorize multiple Box Skills on a single folder.

You can build Box Skills using the Box Skills Kit.

Box Skills perform custom processing for files uploaded to Box.

Box Skills perform custom processing for files uploaded to Box.

How does a Box Skill work?

Box Skills, when enabled by the Box admin, are automatically triggered whenever a file is uploaded to a pre-configured folder in Box.

When the trigger event occurs, the event pump fires an event notification to the Box Skill application.

  • The event pump sends the event to an invocation URL provided as part of registering a Box Skill application with Box. The invocation URL is a web address for the server where the Box Skill application will run.
  • The event payload contains information about the upload event, including information about the file object and its location in Box.
  • The event payload also includes two temporary access tokens that the Box Skill application can use; one to retrieve the file from Box and one to write to its metadata.

The Box Skill application interprets the incoming event and retrieves the file from Box using one of the access tokens in the payload.

The Box Skill application processes the retrieved file. Box Skills can perform any custom processing.

Once the processing is complete, the Box Skill application retrieves the output of the processing.

The Box Skill application adapts the output and writes the data to a metadata template that generates the Box Skills metadata cards using the second access token.

Triggers

A trigger is an event that occurs in Box, such as a file upload. Currently, a Box Skill can be triggered by:

  • File uploaded to a folder
  • File copied to a folder
  • File moved to a folder

Invocation URL

An invocation URL is a HTTP endpoint to invoke a function via a HTTP request. The invocation URL will be specific to a function and the format will vary based on the server provider.

Example:
If you are using Google Cloud Functions, for example, the HTTP endpoint will look like this:

https://YOUR_REGION-YOUR_PROJECT_ID.cloudfunctions.net/FUNCTION_NAME

Providing this URL to Box will allow Box to invoke your function based on a trigger event in Box. When the trigger event occurs, Box will send an event notification to the invocation URL.

Event payload

The event notification payload includes information about the file to be processed and access tokens to access that file and write metadata to the file in Box.

Within the event payload, there are three critical pieces of information that you will use to create a Box Skill:

  • A read access token to retrieve a file from Box
  • A write access token to write metadata to the file object in Box
  • A unique ID of the file in Box that is used to retrieve the file and write to the file’s metadata

Here is a sample event payload:

{
    "type": "skill_invocation",
    "skill": {
        "id": "137",
        "type": "skill",
        "name": "Hello World Skill",
        "api_key": "hxel2s12wd2h9r8ne103c4gjbqefofih"
    },
    "token": {
        "write": {
            "access_token": "REDACTED",
            "expires_in": 1540924150,
            "restricted_to": "[{\"scope\":\"gcm\"},{\"scope\":\"item_upload\",\"object_id\":340708371275,\"object_type\":\"file\"},{\"scope\":\"manage_skill_invocations\"}]",
            "token_type": "bearer"
        },
        "read": {
            "access_token": "REDACTED",
            "expires_in": 1540924150,
            "restricted_to": "[{\"scope\":\"gcm\"},{\"scope\":\"item_read\",\"object_id\":340708371275,\"object_type\":\"file\"}]",
            "token_type": "bearer"
        }
    },
    "status": {
        "state": "invoked",
        "message": "",
        "error_code": "",
        "additional_info": ""
    },
    "id": "fd1d2e53-35f5-41fb-9c25-4ba326daf2f9_341016304",
    "created_at": "2018-10-29T11:29:09-07:00",
    "trigger": "FILE_CONTENT",
    "enterprise": {
        "type": "enterprise",
        "id": "94713934",
        "name": "Important Photo"
    },
    "source": {
        "type": "file",
        "id": "340708371275",
        "name": "200386_main.jpg",
        "sequence_id": "0",
        "file_version": {
            "id": "359740326875"
        },
        "owner_enterprise_id": "94713934",
        "parent": {
            "id": "56598494243"
        },
        "old_parent_id": "56598494243",
        "collab_accessible_by": {
            "type": "",
            "id": "",
            "name": "",
            "login": ""
        },
        "size": 150031
    },
    "event": {
        "event_id": "fd1d2e53-35f5-41fb-9c25-4ba326daf2f9",
        "event_type": "ITEM_UPLOAD",
        "created_at": "2018-10-29T11:29:09-07:00",
        "created_by": {
            "type": "user",
            "id": "6122733934",
            "name": "Box Skills",
            "login": "boxskills-dev@boxdemo.com"
        },
        "source": {
            "type": "file",
            "id": "340708371275",
            "name": "200386_main.jpg",
            "sequence_id": "0",
            "file_version": {
                "id": "359740326875"
            },
            "owner_enterprise_id": "94713934",
            "parent": {
                "id": "56598494243"
            },
            "old_parent_id": "56598494243",
            "collab_accessible_by": {
                "type": "",
                "id": "",
                "name": "",
                "login": ""
            }
        }
    },
    "parameters": {}
}

Access tokens

The event payload provides access tokens so your Box Skill can easily interact with the Box file and metadata APIs. These tokens are downscoped to the specific file that triggered the payload and can only retrieve the file and write to the file’s metadata.

Box Skills metadata cards

Box Skills write metadata to files in Box. The metadata written by a Box Skill can be displayed directly in the Box web application in the right-hand sidebar when a user is previewing a file in Box. The metadata is visualized using a series of pre-designed cards, designed to display some of the common outputs from third-party AI/ML services. To display this information, your Box Skill application can write to a pre-built, globally available metadata template that will dynamically generate the card.

Box Skills can display metadata directly in the Box web application using a series of pre-designed cards.

Box Skills can display metadata directly in the Box web application using a series of pre-designed cards.

Currently, there are four cards available:

  • Topic card - presents a list of keywords (e.g., labels, tags) to be shown in association with a file, optionally with relevant timestamps for when those keywords appear
  • Faces card - displays a set of images (e.g., faces) with their relevant timestamps on a media file
  • Transcript card - displays a transcript with the corresponding timestamps on a media file. This card can also be used to store text without timestamps
  • Status card - displays the statuses and any errors for your Box Skill application

When creating your skill, you can use any combination of these cards, even displaying multiple instances of the same card.


Introducing Box Skills


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.