Ok I see. Now have a different duplidate case to discuss.
Suppose we have a service which reads from a kafka topic and then forwards the event to another topic.
Now, just after reading the event from the topic and after forwarding it to the next topic this service goes down and don't commit/acknowledge the forwarding. In this case, it will read the same message once more and do the operation again creating a duplicate. How should we handle this scenario? Add a db in between to keep track of the events being processed to be able to track the duplicates?
Ok I see. Now have a different duplidate case to discuss.
Suppose we have a service which reads from a kafka topic and then forwards the event to another topic.
Now, just after reading the event from the topic and after forwarding it to the next topic this service goes down and don't commit/acknowledge the forwarding. In this case, it will read the same message once more and do the operation again creating a duplicate. How should we handle this scenario? Add a db in between to keep track of the events being processed to be able to track the duplicates?
Did not get idea with UUID, can you give some example please. Thanks