📙Store Web

Prerequisites

Requires a user auth token. See the API Auth section.

Storing data - API

For now Mindplug handles the storing of text data only. You can store data given the following parameters:

1. Database name: db

2. A collection name: collection

3. A webpage url to crawl and store: url

4. metadata which is any arbitrary object for the content

5. OPTIONAL: chunkSize

   1. Defaults to 1024. Represents the amount of characters stored per embedding. If you give content size of 1300 characters, this will make 2 embeddings of 1k and 300 characters respectively.

Mindplug uses recursive chunking to chunk the string content. Thus the chunkSize will vary slightly across different embeddings.

For any project or collection that does not already exist, it will be auto created for the user without any additional work.

import mindplug from "@/src/mindplugAPI"; // base instance

mindplug.post("/data/store/webpage", {
    db: "walmart",
    collection: "office supplies",
    url: "mindplug.io",
    metadata: {
        lastStock: "June 2023",
        totalItems: 123
    }
});

An upload id and a list of stored vector IDs are returned for later use.

Sample Response

{
    "data": {
        "success": true,
        "vectorIds": [
            "8ad610a8-9dc8-4e3c-bae4-0e10146fd43a"
        ],
        "uploadId": "ea209649-8f38-40e0-9b84-0cdb1809db6c"
    }
}