> For the complete documentation index, see [llms.txt](https://docs.adiom.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.adiom.io/reference/connectors/cosmos-db-nosql.md).

# Cosmos DB NoSQL

The Cosmos DB NoSQL connector is supported both as a source and a destination. It runs as a separate gRPC service that dsync/dsynct connects to, using the Java SDK optimized for bulk operations.

## Running the Connector

Set the Cosmos DB account URL and key. These can be found under "Settings" -> "Keys" in the Azure Portal.

```bash
export COSMOS_URI="..."
export COSMOS_KEY="..."
```

```bash
docker run \
  --network mynet --name cosmosnosqlconnector \
  markadiom/cosmosnosqlconnector 8089 $COSMOS_URI $COSMOS_KEY
```

## gRPC Configuration

The connector exposes a gRPC endpoint. When specifying it as a source or destination, use the `grpc://` scheme with the `--insecure` flag (TLS is not enabled by default):

```
grpc://cosmosnosqlconnector:8089 --insecure
```

When running outside of Docker networking, replace the container name with the appropriate host (e.g. `grpc://localhost:8089`).

## Data Type

The Cosmos DB NoSQL connector uses **JSON**.

## ID Format

The document ID is composed of the shard key followed by the `id` field. The `id` field is always the last part of the ID. If `id` is itself the last path of the shard key, it is not duplicated.

Every ID part is emitted as a string, even when the shard key value is a number or boolean in the document.

For example, with a shard key of `/region`:

| `id`    | `region`    | Document ID          |
| ------- | ----------- | -------------------- |
| `"123"` | `"us-east"` | `["us-east", "123"]` |

With a shard key of `/id` (where `id` is the shard key):

| `id`    | Document ID |
| ------- | ----------- |
| `"123"` | `["123"]`   |

With a hierarchical shard key of `/region/tenant`:

| `id`    | `region`    | `tenant` | Document ID                  |
| ------- | ----------- | -------- | ---------------------------- |
| `"123"` | `"us-east"` | `"acme"` | `["us-east", "acme", "123"]` |

## Emitted Format

This is what the Cosmos DB NoSQL source hands to Dsync, and therefore what a [transformation](/enterprise/running-dsynct/data-transformations.md) receives.

**Namespace:** `database.container`. Database and container names that contain a dot are not supported.

**Initial sync:** each document is emitted as JSON with the Cosmos DB system fields removed: `_rid`, `_self`, `_etag`, `_attachments`, `_ts` and `_lsn` by default (the list can be changed with the `cosmosInternalKeys` setting in the connector's optional YAML config). The `id` field and the shard key fields are kept.

**CDC:** changes are read from the change feed in *all versions and deletes* mode.

| Source operation                                        | Event type | ID                         | Payload                                  |
| ------------------------------------------------------- | ---------- | -------------------------- | ---------------------------------------- |
| create                                                  | `INSERT`   | Shard key parts, then `id` | The full document without system fields. |
| replace (and any operation other than create or delete) | `UPDATE`   | Shard key parts, then `id` | The full document without system fields. |
| delete                                                  | `DELETE`   | Shard key parts, then `id` | None.                                    |

If the connector is started with the environment variable `COSMOS_DISABLE_ALL_VERSIONS_AND_DELETES=true`, it uses the *latest version* change feed mode instead. In that mode every change, including inserts, arrives as an `UPDATE` with the full document, and deletes are not emitted.

### Example

Source document in `shop.orders`, where the container's shard key is `/region`:

```json
{"id": "123", "region": "us-east", "qty": 5, "_rid": "...", "_self": "...", "_etag": "...", "_attachments": "...", "_ts": 1700000000}
```

| Stage                   | ID                     | Payload                                        |
| ----------------------- | ---------------------- | ---------------------------------------------- |
| Initial sync            | (part of the document) | `{"id": "123", "region": "us-east", "qty": 5}` |
| Insert                  | `["us-east", "123"]`   | `{"id": "123", "region": "us-east", "qty": 5}` |
| Update (`qty` set to 6) | `["us-east", "123"]`   | `{"id": "123", "region": "us-east", "qty": 6}` |
| Delete                  | `["us-east", "123"]`   | None                                           |

### Things to Know

* The ID has more than one part unless the shard key is `/id`. A transform must declare the parts with `idkeys`, for example `idkeys: ["region", "id"]`. See [IDs, Data Types and CDC Events](/enterprise/running-dsynct/data-transformations/data-types.md).
* Updates always carry the whole document, so a transform never sees which fields changed.
* Only top-level shard key paths such as `/region` are supported. Nested paths such as `/address/zip` are not.
* The connector supports fetching documents by ID, so it works with `testsync`, Verify IDs and the Sync Tester. Every ID part must be a string when fetching.