Extract metadata from file (structured)

With the Box AI API, you can extract file metadata as key-value pairs. As input, you can create a structure with the fields parameter, or use an existing metadata template. To learn more about creating templates, see customizing metadata templates in the Admin Console or use the . You can also autofill metadata in templates with the Standard Extraction Agent or the Enhanced Extraction Agent.

Supported file formats

The endpoint supports the following file formats:

Documents

PDF
DOC
DOCX
GDOC
ODT
Box Note
TEXT
RTF
XDW
AS

Images

TIFF
TIF
PNG
JPEG
JPG
WEBP

Presentations and spreadsheets

PPT
PPTX
GSLIDE
GSLIDES
ODP
OTP
XLS
XLSX
XLSM
ODS
CSV

Code files

Languages: .js, .py, .css, .php, .sql
JSON
HTML
XML
MD

Box AI applies optical character recognition (OCR) when processing image files and scanned documents. This removes the need to convert images to PDF before extraction.

Supported languages

Box AI can extract metadata from documents in the following languages:

English
Japanese
Chinese
Korean

Cyrillic-based languages (such as Russian, Ukrainian, Bulgarian, and Serbian)

You don’t need additional configuration to use different languages or image formats. Box AI detects the language and applies OCR when needed.

Before you start

Complete the steps in to create a platform app and authenticate.

Send a request

To send a request, use the POST /2.0/ai/extract_structured endpoint.

curl -i -L -X POST 'https://api.box.com/2.0/ai/extract_structured' \
     -H 'content-type: application/json' \
     -H 'authorization: Bearer <ACCESS_TOKEN>' \
     -d '{
        "items": [
          {
            "id": "12345678",
            "type": "file",
            "content": "This is file content."
          }
        ],
        "fields": [
          {
            "key": "name",
            "description": "The name of the person.",
            "displayName": "Name",
            "prompt": "The name is the first and last name from the email address.",
            "type": "string",
            "options": [
              {
                "key": "First Name"
              },
              {
                "key": "Last Name"
              }
            ]
          }
        ],
        "ai_agent": {
          "type": "ai_agent_extract_structured",
          "long_text": {
            "model": "openai__gpt_5_mini"
          },
          "basic_text": {
            "model": "openai__gpt_5_mini"
          }
        }
      }'

await client.ai.createAiExtractStructured({
  fields: [
    {
      key: 'firstName',
      displayName: 'First name',
      description: 'Person first name',
      prompt: 'What is your first name?',
      type: 'string',
    } satisfies AiExtractStructuredFieldsField,
    {
      key: 'lastName',
      displayName: 'Last name',
      description: 'Person last name',
      prompt: 'What is your last name?',
      type: 'string',
    } satisfies AiExtractStructuredFieldsField,
    {
      key: 'dateOfBirth',
      displayName: 'Birth date',
      description: 'Person date of birth',
      prompt: 'What is the date of your birth?',
      type: 'date',
    } satisfies AiExtractStructuredFieldsField,
    {
      key: 'age',
      displayName: 'Age',
      description: 'Person age',
      prompt: 'How old are you?',
      type: 'float',
    } satisfies AiExtractStructuredFieldsField,
    {
      key: 'hobby',
      displayName: 'Hobby',
      description: 'Person hobby',
      prompt: 'What is your hobby?',
      type: 'multiSelect',
      options: [
        { key: 'guitar' } satisfies AiExtractStructuredFieldsOptionsField,
        { key: 'books' } satisfies AiExtractStructuredFieldsOptionsField,
      ],
    } satisfies AiExtractStructuredFieldsField,
  ],
  items: [new AiItemBase({ id: file.id })],
  aiAgent: agentIgnoringOverridingEmbeddingsModel,
} satisfies AiExtractStructured);

client.ai.create_ai_extract_structured(
    [AiItemBase(id=file.id)],
    fields=[
        CreateAiExtractStructuredFields(
            key="firstName",
            display_name="First name",
            description="Person first name",
            prompt="What is your first name?",
            type="string",
        ),
        CreateAiExtractStructuredFields(
            key="lastName",
            display_name="Last name",
            description="Person last name",
            prompt="What is your last name?",
            type="string",
        ),
        CreateAiExtractStructuredFields(
            key="dateOfBirth",
            display_name="Birth date",
            description="Person date of birth",
            prompt="What is the date of your birth?",
            type="date",
        ),
        CreateAiExtractStructuredFields(
            key="age",
            display_name="Age",
            description="Person age",
            prompt="How old are you?",
            type="float",
        ),
        CreateAiExtractStructuredFields(
            key="hobby",
            display_name="Hobby",
            description="Person hobby",
            prompt="What is your hobby?",
            type="multiSelect",
            options=[
                CreateAiExtractStructuredFieldsOptionsField(key="guitar"),
                CreateAiExtractStructuredFieldsOptionsField(key="books"),
            ],
        ),
    ],
    ai_agent=agent_ignoring_overriding_embeddings_model,
)

await client.Ai.CreateAiExtractStructuredAsync(requestBody: new AiExtractStructured(items: Array.AsReadOnly(new [] {new AiItemBase(id: file.Id)})) { Fields = Array.AsReadOnly(new [] {new AiExtractStructuredFieldsField(key: "firstName") { DisplayName = "First name", Description = "Person first name", Prompt = "What is your first name?", Type = "string" },new AiExtractStructuredFieldsField(key: "lastName") { DisplayName = "Last name", Description = "Person last name", Prompt = "What is your last name?", Type = "string" },new AiExtractStructuredFieldsField(key: "dateOfBirth") { DisplayName = "Birth date", Description = "Person date of birth", Prompt = "What is the date of your birth?", Type = "date" },new AiExtractStructuredFieldsField(key: "age") { DisplayName = "Age", Description = "Person age", Prompt = "How old are you?", Type = "float" },new AiExtractStructuredFieldsField(key: "hobby") { DisplayName = "Hobby", Description = "Person hobby", Prompt = "What is your hobby?", Type = "multiSelect", Options = Array.AsReadOnly(new [] {new AiExtractStructuredFieldsOptionsField(key: "guitar"),new AiExtractStructuredFieldsOptionsField(key: "books")}) }}) });

try await client.ai.createAiExtractStructured(requestBody: AiExtractStructured(fields: [AiExtractStructuredFieldsField(key: "firstName", displayName: "First name", description: "Person first name", prompt: "What is your first name?", type: "string"), AiExtractStructuredFieldsField(key: "lastName", displayName: "Last name", description: "Person last name", prompt: "What is your last name?", type: "string"), AiExtractStructuredFieldsField(key: "dateOfBirth", displayName: "Birth date", description: "Person date of birth", prompt: "What is the date of your birth?", type: "date"), AiExtractStructuredFieldsField(key: "age", displayName: "Age", description: "Person age", prompt: "How old are you?", type: "float"), AiExtractStructuredFieldsField(key: "hobby", displayName: "Hobby", description: "Person hobby", prompt: "What is your hobby?", type: "multiSelect", options: [AiExtractStructuredFieldsOptionsField(key: "guitar"), AiExtractStructuredFieldsOptionsField(key: "books")])], items: [AiItemBase(id: file.id)]))

client.getAi().createAiExtractStructured(new AiExtractStructured.Builder(Arrays.asList(new AiItemBase(file.getId()))).fields(Arrays.asList(new AiExtractStructuredFieldsField.Builder("firstName").description("Person first name").displayName("First name").prompt("What is your first name?").type("string").build(), new AiExtractStructuredFieldsField.Builder("lastName").description("Person last name").displayName("Last name").prompt("What is your last name?").type("string").build(), new AiExtractStructuredFieldsField.Builder("dateOfBirth").description("Person date of birth").displayName("Birth date").prompt("What is the date of your birth?").type("date").build(), new AiExtractStructuredFieldsField.Builder("age").description("Person age").displayName("Age").prompt("How old are you?").type("float").build(), new AiExtractStructuredFieldsField.Builder("hobby").description("Person hobby").displayName("Hobby").prompt("What is your hobby?").type("multiSelect").options(Arrays.asList(new AiExtractStructuredFieldsOptionsField("guitar"), new AiExtractStructuredFieldsOptionsField("books"))).build())).aiAgent(agentIgnoringOverridingEmbeddingsModel).build())

BoxAIExtractMetadataTemplate template = new BoxAIExtractMetadataTemplate("templateKey", "enterprise");
BoxAIExtractStructuredResponse result = BoxAI.extractMetadataStructured(
    api,
    Collections.singletonList(new BoxAIItem("123456", BoxAIItem.Type.FILE)),
    template
);
JsonObject sourceJson = result.getSourceJson();

Parameters

The following parameters apply to this request. Mandatory parameters are in bold. The items array must contain exactly one element. For prompt and file limits, see .

Parameter	Description	Example
`metadata_template`	The metadata template containing the fields to extract. For your request to work, you must provide either `metadata_template` or `fields`, but not both.
`metadata_template.type`	The type of metadata template.	`metadata_template`
`metadata_template.scope`	The scope of the metadata template, which can be `global` or `enterprise`. Global templates are available to any Box enterprise. Enterprise templates are bound to a specific enterprise.	`enterprise`
`metadata_template.template_key`	The name of your metadata template.	`invoice`
`items.id`	The Box file ID of the document. The ID must reference a file with an extension.	`1233039227512`
`items.type`	The type of the supplied input.	`file`
`ai_agent`	Overrides the default model configuration. You can change the model, prompt template, system message, or LLM parameters. See for how it works and for examples.
`include_confidence_score`	A flag to indicate whether to include the confidence score for every extracted field.	`true`
`include_reference`	A flag to indicate whether to include references for every extracted field.	`true`
`items.content`	The content of the item, often the text representation.	`This article is about Box AI`.
`fields.description`	A description of the field.	`The person's name.`
`fields.displayName`	The display name of the field.	`Name`
`fields.key`	A unique identifier for the field.	`name`
`fields.namespace`	The namespace of the taxonomy source. Required when you use a `taxonomy` field from an existing taxonomy.	`string`
`fields.options`	A list of options for this field. This is most often used in combination with the `enum` and `multiSelect` field types.	`[{"key":"First Name"},{"key":"Last Name"}]`
`fields.options.key`	A unique identifier for the option.	`First Name`
`fields.prompt`	Additional context about the key (identifier) that can include how to find and format it.	`Name is the first and last name from the email address`
`fields.type`	The type of the field. Types include, but are not limited to, `string`, `float`, `date`, `enum`, `multiSelect`, `struct`, and `table`.	`string`
`fields.taxonomy_key`	The identifier for a taxonomy, which corresponds to the `key` of the taxonomy source. Required when you use a `taxonomy` field.	`string`

`struct` and `table` field types

The Box AI extract_structured API supports two complex field types, struct and table, in addition to the scalar types (string, float, date, enum, and multiSelect). The struct and table types enable you to extract grouped and repeating structured data from documents.

For best results, use the Enhanced Extract Agent.

`struct` field type

Use the struct type to group multiple related sub-fields into a single named JSON object. This is useful when you want to extract a set of related values that belong together and receive them as one structured object rather than separate flat fields. For example, use it for an address or a person’s contact details. A struct field requires a fields array that defines its sub-fields. Each sub-field is an object with the following properties:

key: The unique identifier for the sub-field.
type: The type of the sub-field. Supported types are string, text, number, float, boolean, date, enum, multiSelect, and array[<simple_type>] (for example, array[string]). Nested struct or table types are not supported as sub-fields.
displayName: The display name of the sub-field.
description: A description of the sub-field.
prompt: Additional context about the sub-field that can include how to find and format it.

You can add a prompt at the struct field level when instructions apply to the whole grouped object.

The output is a single JSON object containing the extracted sub-field values. Example request for the struct field type:

{
  "fields": [
    {
      "key": "address",
      "displayName": "Address",
      "type": "struct",
      "fields": [
        { "key": "street_name", "type": "string" },
        { "key": "home_number", "type": "string" },
        { "key": "postal_code", "type": "string" },
        { "key": "city", "type": "string" }
      ]
    }
  ]
}

Response:

{
  "answer": {
    "address": {
      "street_name": "Main St",
      "home_number": "123",
      "postal_code": "94105",
      "city": "San Francisco"
    }
  }
}

`table` field type

Use the table type to extract repeating rows of structured data as an array of JSON objects, where each object represents one row. This is useful when a document contains multiple instances of the same data structure, for example, line items in an invoice or entries in a tax table. A table field requires a fields array that defines the columns (sub-fields) of each row. The sub-field properties and supported types are identical to those of struct.

Table extraction is not limited to visually formatted tables. The table type extracts repeating data from a grid, key-value pairs, a form layout, or plain prose.

The output is an array of JSON objects, where each object represents one extracted row. Example request for the table field type:

{
  "fields": [
    {
      "key": "line_items",
      "displayName": "Line Items",
      "type": "table",
      "fields": [
        { "key": "description", "type": "string" },
        { "key": "quantity", "type": "float" },
        { "key": "amount", "type": "float" }
      ]
    }
  ]
}

Response:

{
  "answer": {
    "line_items": [
      { "description": "Desk", "quantity": 2.0, "amount": 399.99 },
      { "description": "Chair", "quantity": 4.0, "amount": 149.99 }
    ]
  }
}

Supported sub-field types

Both struct and table fields support the following types.

Type	Notes
`string`	Scalar or array[string]
`text`	Scalar or array[text]
`number`	Scalar or array[number]
`float`	Scalar or array[float]
`boolean`	Scalar or array[boolean]
`date`	Scalar or array[date]
`enum`	Single occurrence only
`multiSelect`	Single occurrence only

Nested struct and table types are not supported as sub-fields.

Tutorial: Turn supplier agreements into structured procurement data

See the struct and table field types in action. Extract grouped vendor details and a repeating delivery schedule from a supplier agreement, then map the result to a downstream procurement record.

`include_confidence_score` parameter

The include_confidence_score parameter returns confidence scores for each extracted field. These scores show how likely it is that the extracted information is correct. Box provides both a number score and a simple rating (Low, Medium, or High) so you and automated tools can quickly see how reliable the results are. To calculate confidence, Box generates multiple answers for the same request in slightly different ways (such as changing settings, rephrasing prompts, or rearranging fields) and evaluates the consistency and frequency of the returned values. Box then turns these signals into probabilities that reflect actual accuracy.

High confidence (0.90 or higher) means the value is very likely correct, and little or no human review is needed.
Medium confidence (0.70 – 0.89) means the value is likely correct, and a quick check is recommended, depending on your risk tolerance.
Low confidence (below 0.70) means the value is uncertain, and a human review is strongly recommended.

`include_reference` parameter

The include_reference parameter returns supporting citations and bounding boxes for extracted values. These help you validate extracted values against the source document. A reference is a short text excerpt from the source file that supports an extracted value. A bounding box identifies the region on the page where that supporting text is located. Box AI calculates bounding boxes from the page-based OCR representation and returns them only for files supported by OCR. To include references in the response, set include_reference to true.

{
  "items": [...],
  "fields": [...],
  "include_reference": true
}

Response structure

The response includes:

answer: The extracted metadata fields.
references: Supporting citations for each extracted field.

Each reference contains:

item_id: The source file ID.
page: The zero-indexed page number.
bounding_box: The location of the cited text, when available.
text: The cited text excerpt.

Response schema example

{
  "answer": {
    "vendor": "Quasar Innovations",
    "document_total": 1050,
    "document_type": "Purchase Order"
  },
  "references": {
    "document_type": [
      {
        "item_id": "314",
        "page": 0,
        "bounding_box": {
          "left": 0.1,
          "top": 0.1,
          "right": 0.3,
          "bottom": 0.5
        },
        "text": "Vendor Company: Acme"
      }
    ],
    "contract": [
      {
        "item_id": "314",
        "page": 0,
        "bounding_box": {
          "left": 0.1,
          "top": 0.1,
          "right": 0.3,
          "bottom": 0.5
        },
        "text": "For: $1234"
      }
    ],
    "start_date": [
      {
        "item_id": "314",
        "page": 0,
        "bounding_box": {
          "left": 0.1,
          "top": 0.1,
          "right": 0.3,
          "bottom": 0.5
        },
        "text": "Contracted on: 2023-12-02"
      }
    ],
    "services": [
      [
        {
          "item_id": "314",
          "page": 1,
          "bounding_box": {
            "left": 0.1,
            "top": 0.1,
            "right": 0.3,
            "bottom": 0.5
          },
          "text": "Cloud Computing"
        }
      ],
      [
        {
          "item_id": "314",
          "page": 1,
          "bounding_box": {
            "left": 0.1,
            "top": 0.1,
            "right": 0.3,
            "bottom": 0.5
          },
          "text": "Data Analytics"
        }
      ],
      [
        {
          "item_id": "314",
          "page": 1,
          "bounding_box": {
            "left": 0.1,
            "top": 0.1,
            "right": 0.3,
            "bottom": 0.5
          },
          "text": "Technical Support"
        }
      ]
    ]
  }
}

Bounding box format

A bounding box is an object with four normalized coordinates:

left: The normalized x-coordinate of the left edge of the bounding box.
top: The normalized y-coordinate of the top edge of the bounding box.
right: The normalized x-coordinate of the right edge of the bounding box.
bottom: The normalized y-coordinate of the bottom edge of the bounding box.

These values range from 0 to 1, where (0,0) is the top-left corner and (1,1) is the bottom-right corner of the page. For example:

{
  "left": 0.20,
  "top": 0.10,
  "right": 0.60,
  "bottom": 0.30
}

Every extracted metadata field has an associated list of references. For multi-value fields, each extracted value has its own list of references, resulting in a nested list structure. bounding_box is optional and can be null.

Use cases

This example shows you how to extract metadata from a sample invoice. Extract the vendor name, invoice number, and a few more details.

Invoice for Quasar Innovations with a vendor, invoice number, purchase order number, two line items, and a total of $1,050

Create the request

To get the response from Box AI, call the POST /2.0/ai/extract_structured endpoint with the following parameters:

items.type and items.id to specify the file to extract data from
fields to specify the data to extract from the file
metadata_template to supply an existing metadata template

You can use either fields or metadata_template to specify your structure, but not both.

Use `fields` parameter

The fields parameter enables you to specify the data you want to extract. Each fields object includes parameters you can use to describe that data. For example, you can add the field type, description, or a prompt with additional context.

curl -i -L -X POST 'https://api.box.com/2.0/ai/extract_structured' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <ACCESS_TOKEN>' \
--data '{
    "items": [
        {
            "id": "1517628697289",
            "type": "file"
        }
    ],
    "fields": [
        {
            "key": "document_type",
            "type": "enum",
            "prompt": "What type of document is this?",
            "options": [
                {
                    "key": "Invoice"
                },
                {
                    "key": "Purchase Order"
                },
                {
                    "key": "Unknown"
                }
            ]
        },
        {
            "key": "document_date",
            "type": "date"
        },
        {
            "key": "vendor",
            "description": "The name of the entity.",
            "prompt": "Which vendor is sending this document?",
            "type": "string"
        },
        {
            "key": "document_total",
            "type": "float"
        }
    ]
  }'

The response lists the specified fields and their values:

{
    "document_date": "2024-02-13",
    "vendor": "Quasar Innovations",
    "document_total": 1050,
    "document_type": "Purchase Order"
}

Use metadata template

If you prefer to use a metadata template, you can provide its template_key, type, and scope.

curl -i -L -X POST 'https://api.box.com/2.0/ai/extract_structured' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <ACCESS_TOKEN>' \
--data '{
    "items": [
        {
            "id": "1517628697289",
            "type": "file"
        }
    ],
    "metadata_template": {
        "template_key": "rbInvoicePO",
        "type": "metadata_template",
        "scope": "enterprise_1134207681"
    }
}'

The response lists the fields included in the metadata template and their values:

{
  "documentDate": "February 13, 2024",
  "total": "$1050",
  "documentType": "Purchase Order",
  "vendor": "Quasar Innovations",
  "purchaseOrderNumber": "003"
}

Enhanced Extract Agent

To use the Enhanced Extract Agent, specify the ai_agent object as follows:

{
  "ai_agent": {
    "type": "ai_agent_id",
    "id": "enhanced_extract_agent"
  }
}

To extract data using the Enhanced Extract Agent, you need one of the following:

Inline field definitions (best when fields change frequently)
Metadata template (best when fields stay consistent)

See the following sample, which uses the Box Python SDK:

from box_sdk_gen import (
    AiAgentReference,
    AiAgentReferenceTypeField,
    AiItemBase,
    AiItemBaseTypeField,
    BoxClient,
    BoxCCGAuth,
    CCGConfig,
    CreateAiExtractStructuredMetadataTemplate
)

# Create your client credentials grant config from the Developer Console
ccg_config = CCGConfig(
    client_id="my_box_client_id", # replace with your client ID
    client_secret="my_box_client_secret", # replace with your client secret
    user_id="my_box_user_id", # replace with the Box user ID that has access
                              # to the file you are referencing
)
auth = BoxCCGAuth(config=ccg_config)
client = BoxClient(auth=auth)
# Create the agent config referencing the Enhanced Extract Agent
enhanced_extract_agent_config = AiAgentReference(
    id="enhanced_extract_agent",
    type=AiAgentReferenceTypeField.AI_AGENT_ID
)
# Use the Box SDK to call the extract_structured endpoint
box_ai_response = client.ai.create_ai_extract_structured(
    # Create the items array containing the file information to extract from
    items=[
        AiItemBase(
            id="my_box_file_id", # replace with the file ID
            type=AiItemBaseTypeField.FILE
        )
    ],
    # Reference the Box metadata template
    metadata_template=CreateAiExtractStructuredMetadataTemplate(
        template_key="InvoicePO",
        scope="enterprise"
    ),
    # Attach the agent config you created earlier
    ai_agent=enhanced_extract_agent_config,
)
print(f"box_ai_response: {box_ai_response.answer}")

Tutorial: Automate invoice intake with Box AI Extract

See structured extraction in action. Build an end-to-end automation that watches a folder, extracts invoice fields, and writes metadata back to each file.

​Supported file formats

​Supported languages

​Before you start

​Send a request

​Parameters

​struct and table field types

​struct field type

​table field type

​Supported sub-field types

Tutorial: Turn supplier agreements into structured procurement data

​include_confidence_score parameter

​include_reference parameter

​Response structure

​Response schema example

​Bounding box format

​Use cases

​Create the request

​Use fields parameter

​Use metadata template

​Enhanced Extract Agent

Tutorial: Automate invoice intake with Box AI Extract

Supported file formats

Supported languages

Before you start

Send a request

Parameters

`struct` and `table` field types

`struct` field type

`table` field type

Supported sub-field types

`include_confidence_score` parameter

`include_reference` parameter

Response structure

Response schema example

Bounding box format

Use cases

Create the request

Use `fields` parameter

Use metadata template

Enhanced Extract Agent