python-kafka consumer

 

 

1단계 - pip install 

pip install kafka-python

 

 

2단계 - kafka consumer python

from kafka import KafkaConsumer
from json import loads
 
# topic, broker list
consumer = KafkaConsumer(
    'test',
    bootstrap_servers=['my-kafka.default.svc.cluster.local:9092'],
    enable_auto_commit=True,
)
 
# consumer list를 가져온다
print('[begin] get consumer list')
for message in consumer:
    print("Topic: %s, Partition: %d, Offset: %d, Key: %s, Value: %s" \
    % ( message.topic,
        message.partition,
        message.offset,
        message.key,
        message.value ))
 
print('[end] get consumer list')

 

3단계 - SSL 인증서 추가하기

from kafka import KafkaConsumer
from json import loads
 
# topic, broker list
caRootLocation='CARoot.pem'

consumer = KafkaConsumer(
    'test',
    bootstrap_servers=['my-kafka.default.svc.cluster.local:9092'],
    enable_auto_commit=True,
    security_protocol='SSL',
    ssl_check_hostname=True,
    ssl_cafile=caRootLocation,
 )
 
# consumer list를 가져온다
print('[begin] get consumer list')
for message in consumer:
    print("Topic: %s, Partition: %d, Offset: %d, Key: %s, Value: %s" \
    % ( message.topic,
        message.partition,
        message.offset,
        message.key,
        message.value ))
 
print('[end] get consumer list')

 

 

🧔참고한곳

• https://dev.to/adityakanekar/connecting-to-kafka-cluster-using-ssl-with-python-k2e

Connecting to Kafka cluster using SSL with Python