Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
240 changes: 240 additions & 0 deletions docs/1-using/5-connectors/7-list/37-kinesis.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,240 @@
---
IMPORTANT: This file was generated using src/connectorgen/main.go. DO NOT EDIT.

title: "kinesis"
description: "A Conduit Connector for AWS Kinesis Data Streaming"
---

import ReactDiffViewer from 'react-diff-viewer-continued';
import Chip from '@mui/material/Chip';
import Box from "@mui/system/Box";
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import GitHubIcon from '@mui/icons-material/GitHub';
import IconButton from '@mui/material/IconButton';
import Stack from '@mui/material/Stack';
import Link from '@mui/material/Link';
import StarIcon from '@mui/icons-material/Star';
import Tooltip from '@mui/material/Tooltip';

# kinesis

<Box sx={{
display: 'flex',
alignItems: 'center',
gap: 2,
marginBottom: 4,
borderRadius: 1,
padding: 2,
backgroundColor: 'rgba(0, 0, 0, 0.03)'
}}>
{/* Author info */}
<Box sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
<span>Author: Meroxa, Inc.</span>
<Tooltip title="Created by the Conduit team">
<img src='/img/conduit/conduit-ring.png' width='18' alt="Conduit team logo" />
</Tooltip>
</Box>

{/* Divider */}
<Box sx={{ borderLeft: '1px solid rgba(0, 0, 0, 0.12)', height: 24 }} />

{/* GitHub link */}
<Link
href="https://github.com/conduitio-labs/conduit-connector-kinesis"
target="_blank"
sx={{
display: 'flex',
alignItems: 'center',
gap: 0.5,
color: 'inherit',
textDecoration: 'none',
'&:hover': {
textDecoration: 'underline'
}
}}
>
<GitHubIcon sx={{ fontSize: '1.5rem' }} />
</Link>

{/* Stars */}
<Chip
icon={<StarIcon sx={{ fontSize: '1rem' }} />}
label="0"
size="small"
sx={{ height: 24 }}
/>
</Box>

## Latest release

- [conduit-connector-kinesis_0.2.1_Darwin_arm64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-kinesis/releases/download/v0.2.1/conduit-connector-kinesis_0.2.1_Darwin_arm64.tar.gz)
- [conduit-connector-kinesis_0.2.1_Darwin_x86_64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-kinesis/releases/download/v0.2.1/conduit-connector-kinesis_0.2.1_Darwin_x86_64.tar.gz)
- [conduit-connector-kinesis_0.2.1_Linux_arm64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-kinesis/releases/download/v0.2.1/conduit-connector-kinesis_0.2.1_Linux_arm64.tar.gz)
- [conduit-connector-kinesis_0.2.1_Linux_i386.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-kinesis/releases/download/v0.2.1/conduit-connector-kinesis_0.2.1_Linux_i386.tar.gz)
- [conduit-connector-kinesis_0.2.1_Linux_x86_64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-kinesis/releases/download/v0.2.1/conduit-connector-kinesis_0.2.1_Linux_x86_64.tar.gz)
- [conduit-connector-kinesis_0.2.1_Windows_arm64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-kinesis/releases/download/v0.2.1/conduit-connector-kinesis_0.2.1_Windows_arm64.tar.gz)
- [conduit-connector-kinesis_0.2.1_Windows_i386.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-kinesis/releases/download/v0.2.1/conduit-connector-kinesis_0.2.1_Windows_i386.tar.gz)
- [conduit-connector-kinesis_0.2.1_Windows_x86_64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-kinesis/releases/download/v0.2.1/conduit-connector-kinesis_0.2.1_Windows_x86_64.tar.gz)
- [kinesis_v0.2.1_darwin_amd64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-kinesis/releases/download/v0.2.1/kinesis_v0.2.1_darwin_amd64.tar.gz)
- [kinesis_v0.2.1_darwin_amd64.tar.gz.sigstore.json](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-kinesis/releases/download/v0.2.1/kinesis_v0.2.1_darwin_amd64.tar.gz.sigstore.json)
- [kinesis_v0.2.1_darwin_arm64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-kinesis/releases/download/v0.2.1/kinesis_v0.2.1_darwin_arm64.tar.gz)
- [kinesis_v0.2.1_darwin_arm64.tar.gz.sigstore.json](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-kinesis/releases/download/v0.2.1/kinesis_v0.2.1_darwin_arm64.tar.gz.sigstore.json)
- [kinesis_v0.2.1_linux_amd64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-kinesis/releases/download/v0.2.1/kinesis_v0.2.1_linux_amd64.tar.gz)
- [kinesis_v0.2.1_linux_amd64.tar.gz.sigstore.json](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-kinesis/releases/download/v0.2.1/kinesis_v0.2.1_linux_amd64.tar.gz.sigstore.json)
- [kinesis_v0.2.1_linux_arm64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-kinesis/releases/download/v0.2.1/kinesis_v0.2.1_linux_arm64.tar.gz)
- [kinesis_v0.2.1_linux_arm64.tar.gz.sigstore.json](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-kinesis/releases/download/v0.2.1/kinesis_v0.2.1_linux_arm64.tar.gz.sigstore.json)

## Description

A source and destination connector for AWS Kinesis Data Streaming

## Source Parameters

```yaml
version: 2.2
pipelines:
- id: example
status: running
connectors:
- id: example-source
type: source
plugin: "kinesis"
name: example-source
settings:
# The amazon access key id
# Type: string
aws.accessKeyId: ""
# The region where the stream is hosted
# Type: string
aws.region: ""
# The amazon secret access key
# Type: string
aws.secretAccessKey: ""
# The name of the Kinesis Data Stream.
# Type: string
streamName: ""
# The URL for endpoint override - testing/dry-run only
# Type: string
aws.url: ""
# If true, sets the iterator type to LATEST (iterates from the point
# that the connection begins = CDC). Otherwise it sets the iterator
# type to TRIM_HORIZON (iterates from the oldest record in the shard =
# snapshot). Iterators eventually shift to latest after snapshot has
# been written.
# Type: bool
startFromLatest: ""
# Maximum delay before an incomplete batch is read from the source.
# Type: duration
sdk.batch.delay: "0"
# Maximum size of batch before it gets read from the source.
# Type: int
sdk.batch.size: "0"
# Specifies whether to use a schema context name. If set to false, no
# schema context name will be used, and schemas will be saved with the
# subject name specified in the connector (not safe because of name
# conflicts).
# Type: bool
sdk.schema.context.enabled: "true"
# Schema context name to be used. Used as a prefix for all schema
# subject names. If empty, defaults to the connector ID.
# Type: string
sdk.schema.context.name: ""
# Whether to extract and encode the record key with a schema.
# Type: bool
sdk.schema.extract.key.enabled: "true"
# The subject of the key schema. If the record metadata contains the
# field "opencdc.collection" it is prepended to the subject name and
# separated with a dot.
# Type: string
sdk.schema.extract.key.subject: "key"
# Whether to extract and encode the record payload with a schema.
# Type: bool
sdk.schema.extract.payload.enabled: "true"
# The subject of the payload schema. If the record metadata contains
# the field "opencdc.collection" it is prepended to the subject name
# and separated with a dot.
# Type: string
sdk.schema.extract.payload.subject: "payload"
# The type of the payload schema.
# Type: string
sdk.schema.extract.type: "avro"
```

## Destination Parameters

```yaml
version: 2.2
pipelines:
- id: example
status: running
connectors:
- id: example-destination
type: destination
plugin: "kinesis"
name: example-destination
settings:
# The amazon access key id
# Type: string
aws.accessKeyId: ""
# The region where the stream is hosted
# Type: string
aws.region: ""
# The amazon secret access key
# Type: string
aws.secretAccessKey: ""
# Makes the destination create a new stream if it does not exist when
# writing records.
# Type: bool
auto.create.streams: "true"
# The URL for endpoint override - testing/dry-run only
# Type: string
aws.url: ""
# Accepts a go template as an argument, with the record being written
# as the main data context. If an empty template is passed, the
# partition key will adopt the record key string value. If the record
# key string exceeds 256 it will be trimmed down from start to fit the
# partition key size
# Type: string
partitionKeyTemplate: ""
# The name of the Kinesis Data Stream. It can contain a [Go
# template](https://pkg.go.dev/text/template) that will be executed
# for each record to determine the stream name. By default, the stream
# name is the value of the `opencdc.collection` metadata field.
# Type: string
streamName: "{{ index .Metadata "opencdc.collection" }}"
# Maximum delay before an incomplete batch is written to the
# destination.
# Type: duration
sdk.batch.delay: "0"
# Maximum size of batch before it gets written to the destination.
# Type: int
sdk.batch.size: "0"
# Allow bursts of at most X records (0 or less means that bursts are
# not limited). Only takes effect if a rate limit per second is set.
# Note that if `sdk.batch.size` is bigger than `sdk.rate.burst`, the
# effective batch size will be equal to `sdk.rate.burst`.
# Type: int
sdk.rate.burst: "0"
# Maximum number of records written per second (0 means no rate
# limit).
# Type: float
sdk.rate.perSecond: "0"
# The format of the output record. See the Conduit documentation for a
# full list of supported formats
# (https://conduit.io/docs/using/connectors/configuration-parameters/output-format).
# Type: string
sdk.record.format: "opencdc/json"
# Options to configure the chosen output record format. Options are
# normally key=value pairs separated with comma (e.g.
# opt1=val2,opt2=val2), except for the `template` record format, where
# options are a Go template.
# Type: string
sdk.record.format.options: ""
# Whether to extract and decode the record key with a schema.
# Type: bool
sdk.schema.extract.key.enabled: "true"
# Whether to extract and decode the record payload with a schema.
# Type: bool
sdk.schema.extract.payload.enabled: "true"
```
24 changes: 16 additions & 8 deletions docs/1-using/5-connectors/7-list/39-mongo.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -68,14 +68,22 @@ import Tooltip from '@mui/material/Tooltip';

## Latest release

- [conduit-connector-mongo_0.3.0_Darwin_arm64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.0/conduit-connector-mongo_0.3.0_Darwin_arm64.tar.gz)
- [conduit-connector-mongo_0.3.0_Darwin_x86_64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.0/conduit-connector-mongo_0.3.0_Darwin_x86_64.tar.gz)
- [conduit-connector-mongo_0.3.0_Linux_arm64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.0/conduit-connector-mongo_0.3.0_Linux_arm64.tar.gz)
- [conduit-connector-mongo_0.3.0_Linux_i386.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.0/conduit-connector-mongo_0.3.0_Linux_i386.tar.gz)
- [conduit-connector-mongo_0.3.0_Linux_x86_64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.0/conduit-connector-mongo_0.3.0_Linux_x86_64.tar.gz)
- [conduit-connector-mongo_0.3.0_Windows_arm64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.0/conduit-connector-mongo_0.3.0_Windows_arm64.tar.gz)
- [conduit-connector-mongo_0.3.0_Windows_i386.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.0/conduit-connector-mongo_0.3.0_Windows_i386.tar.gz)
- [conduit-connector-mongo_0.3.0_Windows_x86_64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.0/conduit-connector-mongo_0.3.0_Windows_x86_64.tar.gz)
- [conduit-connector-mongo_0.3.1_Darwin_arm64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.1/conduit-connector-mongo_0.3.1_Darwin_arm64.tar.gz)
- [conduit-connector-mongo_0.3.1_Darwin_x86_64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.1/conduit-connector-mongo_0.3.1_Darwin_x86_64.tar.gz)
- [conduit-connector-mongo_0.3.1_Linux_arm64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.1/conduit-connector-mongo_0.3.1_Linux_arm64.tar.gz)
- [conduit-connector-mongo_0.3.1_Linux_i386.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.1/conduit-connector-mongo_0.3.1_Linux_i386.tar.gz)
- [conduit-connector-mongo_0.3.1_Linux_x86_64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.1/conduit-connector-mongo_0.3.1_Linux_x86_64.tar.gz)
- [conduit-connector-mongo_0.3.1_Windows_arm64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.1/conduit-connector-mongo_0.3.1_Windows_arm64.tar.gz)
- [conduit-connector-mongo_0.3.1_Windows_i386.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.1/conduit-connector-mongo_0.3.1_Windows_i386.tar.gz)
- [conduit-connector-mongo_0.3.1_Windows_x86_64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.1/conduit-connector-mongo_0.3.1_Windows_x86_64.tar.gz)
- [mongo_v0.3.1_darwin_amd64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.1/mongo_v0.3.1_darwin_amd64.tar.gz)
- [mongo_v0.3.1_darwin_amd64.tar.gz.sigstore.json](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.1/mongo_v0.3.1_darwin_amd64.tar.gz.sigstore.json)
- [mongo_v0.3.1_darwin_arm64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.1/mongo_v0.3.1_darwin_arm64.tar.gz)
- [mongo_v0.3.1_darwin_arm64.tar.gz.sigstore.json](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.1/mongo_v0.3.1_darwin_arm64.tar.gz.sigstore.json)
- [mongo_v0.3.1_linux_amd64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.1/mongo_v0.3.1_linux_amd64.tar.gz)
- [mongo_v0.3.1_linux_amd64.tar.gz.sigstore.json](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.1/mongo_v0.3.1_linux_amd64.tar.gz.sigstore.json)
- [mongo_v0.3.1_linux_arm64.tar.gz](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.1/mongo_v0.3.1_linux_arm64.tar.gz)
- [mongo_v0.3.1_linux_arm64.tar.gz.sigstore.json](https://conduit.gateway.scarf.sh/connector/download/conduitio-labs/conduit-connector-mongo/releases/download/v0.3.1/mongo_v0.3.1_linux_arm64.tar.gz.sigstore.json)

## Description

Expand Down
Loading