Building Box Skills

Box Skills is in a Private Beta

The Box Skills framework and developer toolkit are currently available in a closed private beta and is subject to usability and performance issues as well as breaking changes without notice. During this period, Box Skills is not recommended for production use.

Master Beta Agreement required

In order to use any beta software from Box, your company must have a signed Master Beta Agreement (MBA) in place.

Please contact your Box Sales Representative to receive the MBA for signature.

Quickstart

Choose a language to get started:

What is a Box Skill?

A Box Skill is a type of application that performs custom processing for files uploaded to Box.

Often, Box Skills leverage third-party AI/ML services to automatically extract information from files upon upload to Box. For example, a Box Skill could automatically label objects in images using a computer vision service.

The framework allows a Box Skill to receive information about a file as it’s uploaded to Box, retrieve the file for processing, and write to the file’s metadata in Box. Metadata applied by a Box Skill can be displayed to end users via the Box application and can drive other capabilities of the Box platform, such as search, workflow, and policies.

Box Skills are enabled in a Box environment by the primary Box admin. The admin can define which folder(s) a Skill can process. The admin can authorize multiple Box Skills on a single folder.

You can build Box Skills using the Box Skills Kit.

Box Skills perform custom processing for files uploaded to Box.

Box Skills perform custom processing for files uploaded to Box.

How does a Box Skill work?

Box Skills, when enabled by the Box admin, are automatically triggered whenever a file is uploaded to a pre-configured folder in Box.

When the trigger event occurs, the event pump fires an event notification to the Box Skill application.

  • The event pump sends the event to an invocation URL provided as part of registering a Box Skill application with Box. The invocation URL is a web address for the server where the Box Skill application will run.
  • The event payload contains information about the upload event, including information about the file object and its location in Box.
  • The event payload also includes two temporary access tokens that the Box Skill application can use; one to retrieve the file from Box and one to write to its metadata.

The Box Skill application interprets the incoming event and retrieves the file from Box using one of the access tokens in the payload.

The Box Skill application processes the retrieved file. Box Skills can perform any custom processing.

Once the processing is complete, the Box Skill application retrieves the output of the processing.

The Box Skill application adapts the output and writes the data to a metadata template that generates the Box Skills metadata cards using the second access token.

Triggers

A trigger is an event that occurs in Box, such as a file upload. Currently, a Box Skill can be triggered by:

  • File uploaded to a folder
  • File copied to a folder
  • File moved to a folder

Event payload

When the trigger event occurs, the event pump fires an event notification to the Box Skill application. The event notification payload includes information about the file to be processed and access tokens to access that file and write metadata to the file in Box.

Within the event payload, there are three critical pieces of information that you will use to create a Box Skill:

  • A read access token to retrieve a file from Box
  • A write access token to write metadata to the file object in Box
  • A unique ID of the file in Box that is used to retrieve the file and write to the file’s metadata

Here is a sample event payload:

{
    "type": "skill_invocation",
    "skill": {
        "id": "137",
        "type": "skill",
        "name": "Hello World Skill",
        "api_key": "hxel2s12wd2h9r8ne103c4gjbqefofih"
    },
    "token": {
        "write": {
            "access_token": "REDACTED",
            "expires_in": 1540924150,
            "restricted_to": "[{\"scope\":\"gcm\"},{\"scope\":\"item_upload\",\"object_id\":340708371275,\"object_type\":\"file\"},{\"scope\":\"manage_skill_invocations\"}]",
            "token_type": "bearer"
        },
        "read": {
            "access_token": "REDACTED",
            "expires_in": 1540924150,
            "restricted_to": "[{\"scope\":\"gcm\"},{\"scope\":\"item_read\",\"object_id\":340708371275,\"object_type\":\"file\"}]",
            "token_type": "bearer"
        }
    },
    "status": {
        "state": "invoked",
        "message": "",
        "error_code": "",
        "additional_info": ""
    },
    "id": "fd1d2e53-35f5-41fb-9c25-4ba326daf2f9_341016304",
    "created_at": "2018-10-29T11:29:09-07:00",
    "trigger": "FILE_CONTENT",
    "enterprise": {
        "type": "enterprise",
        "id": "94713934",
        "name": "Important Photo"
    },
    "source": {
        "type": "file",
        "id": "340708371275",
        "name": "200386_main.jpg",
        "sequence_id": "0",
        "file_version": {
            "id": "359740326875"
        },
        "owner_enterprise_id": "94713934",
        "parent": {
            "id": "56598494243"
        },
        "old_parent_id": "56598494243",
        "collab_accessible_by": {
            "type": "",
            "id": "",
            "name": "",
            "login": ""
        },
        "size": 150031
    },
    "event": {
        "event_id": "fd1d2e53-35f5-41fb-9c25-4ba326daf2f9",
        "event_type": "ITEM_UPLOAD",
        "created_at": "2018-10-29T11:29:09-07:00",
        "created_by": {
            "type": "user",
            "id": "6122733934",
            "name": "Box Skills",
            "login": "boxskills-dev@boxdemo.com"
        },
        "source": {
            "type": "file",
            "id": "340708371275",
            "name": "200386_main.jpg",
            "sequence_id": "0",
            "file_version": {
                "id": "359740326875"
            },
            "owner_enterprise_id": "94713934",
            "parent": {
                "id": "56598494243"
            },
            "old_parent_id": "56598494243",
            "collab_accessible_by": {
                "type": "",
                "id": "",
                "name": "",
                "login": ""
            }
        }
    },
    "parameters": {}
}

Access tokens

The event payload provides access tokens so your Box Skill can easily interact with the Box file and metadata APIs. These tokens are downscoped to the specific file that triggered the payload and can only retrieve the file and write to the file’s metadata.

Box Skills metadata cards

Box Skills write metadata to files in Box. The metadata written by a Box Skill can be displayed directly in the Box web application in the right-hand sidebar when a user is previewing a file in Box. The metadata is visualized using a series of pre-designed cards, designed to display some of the common outputs from third-party AI/ML services. To display this information, your Box Skill application can write to a pre-built, globally available metadata template that will dynamically generate the card.

Box Skills can display metadata directly in the Box web application using a series of pre-designed cards.

Box Skills can display metadata directly in the Box web application using a series of pre-designed cards.

Currently, there are four cards available:

  • Topic card - presents a list of keywords (e.g., labels, tags) to be shown in association with a file
  • Timeline card - displays a set of images (e.g., faces) or words (e.g., labels or sentiments) with their relevant timestamps on a media file
  • Transcript card - displays a transcript with the corresponding timestamps on a media file. This card can also be used to store text without timestamps
  • Status card - displays the statuses and any errors for your Box Skill application

When creating your skill, you can use any combination of these cards, even displaying multiple instances of the same card.