overview
Confluent Cloud is a fully managed, cloud-native event streaming platform powered by Apache Kafka®. It enables organizations to process and react to data in real-time without managing infrastructure. This guide covers everything you need to get started.
what is confluent cloud?
- Fully managed Apache Kafka® in the cloud (AWS, GCP, Azure)
- Prebuilt connectors for popular data sources and sinks
- Built-in security, monitoring, and scaling
- Integrations with enterprise and SaaS systems
pre-requisites
- Valid business or personal email address
- Internet access and a modern browser (Chrome, Firefox, Edge)
- (Optional) Account with AWS, GCP, or Azure for advanced integrations
- (Recommended) Basic familiarity with Kafka (topics, producers, consumers)
signing up
- Go to https://confluent.cloud
- Click Sign Up in the top-right corner
- Fill out your email, name, and password
- Accept the terms and privacy policy
- Check your inbox and verify your email
- Log in to your new account
navigating confluent cloud
- Dashboard: At-a-glance view of clusters, usage, and recent activity
- Clusters: Create and manage Kafka clusters
- Topics: Create and manage Kafka topics
- Connectors: Integrate with external data systems
- API Keys: Manage credentials for connecting applications
- Security: Control user access, roles, networking
- Billing: Review usage and payment info
creating a kafka cluster
- Click Clusters on the sidebar
- Click Create cluster
- Choose your cloud provider (e.g. AWS, GCP, Azure)
- Select a region close to your systems/users
- Select the cluster type (Basic, Standard, or Dedicated)
- Name your cluster and review settings
- Click Launch cluster
- Wait a few minutes for provisioning to finish
creating a topic
- Click on your cluster, then Topics
- Click Create topic
- Enter a topic name (ex.
user-signups) - Leave default partition and replication settings (or customize if needed)
- Click Create
producing and consuming data
- Go to API Keys and create a new key/secret
- Download these credentials and keep them secure
- Use your favorite Kafka client (Java, Python, .NET, etc.) with the provided bootstrap server and credentials
- Example (Python
confluent-kafka):
from confluent_kafka import Producer
conf = {
'bootstrap.servers': '<broker-endpoint>',
'security.protocol': 'SASL_SSL',
'sasl.mechanism': 'PLAIN',
'sasl.username': '<api-key>',
'sasl.password': '<api-secret>'
}
p = Producer(conf)
p.produce('test-topic', value='hello world')
p.flush()
securing your cluster
- Use API keys for all data-producing/consuming clients
- Assign users and service accounts to specific roles (RBAC)
- All data traffic is encrypted (TLS in transit)
- Restrict public access with VPC peering, PrivateLink, or IP allowlists
- Enable audit logging for compliance and visibility
monitoring and metrics
- Click Metrics for dashboards of traffic, consumer lag, storage, and errors
- Integrate with Datadog, Prometheus, or others for advanced monitoring
- Set alerts for key thresholds (lag, throughput, etc.)
billing and usage
- Access the Billing & Usage section for real-time and historical usage
- Review invoices, payment methods, and cost details
- Learn about quotas and optimize for cost savings at Confluent Billing Docs
troubleshooting
common problems and tips
-
Cannot connect to cluster?
- Double-check your API key and secret
- Verify bootstrap server and region settings
- Confirm any firewall settings allow traffic
-
Authentication error?
- Reset/recreate your API key if unsure
- Keys may take a minute to be fully active
-
Consumer lag or slow performance?
- Check client configuration
- Use more partitions or consumers for scaling
-
UI not updating?
- Refresh your browser, or try clearing the cache
frequently asked questions
-
Can I migrate data from another Kafka cluster?
Yes. Use tools like Cluster Linking, Replicator, or MirrorMaker 2. -
Is my data secure?
Yes. Confluent Cloud encrypts data in transit and at rest, and supports RBAC. -
How do I raise cluster size limits?
For Standard/Dedicated clusters, contact Confluent support. -
What happens if I lose my API key?
Delete the lost key and generate a new one.
further resources
Need more help? Use the Zendesk chat widget or submit a support ticket and our team will assist you!
Comments
0 comments
Please sign in to leave a comment.