{"id":2918,"date":"2024-08-24T12:08:40","date_gmt":"2024-08-24T11:08:40","guid":{"rendered":"https:\/\/contentlabstudy.com\/soft\/?p=2918"},"modified":"2026-09-24T21:15:35","modified_gmt":"2026-09-24T20:15:35","slug":"message-broker-functions","status":"publish","type":"post","link":"https:\/\/contentlabstudy.com\/soft\/message-broker-functions\/","title":{"rendered":"Message Broker functions"},"content":{"rendered":"<p>A <strong>message broker<\/strong> is a process that accepts messages, stores them (at least briefly), and delivers them to consumers. You use one to decouple <em>when<\/em> work happens from <em>who<\/em> asked. It is not a database, not a REST bus, and not \u201casync\u201d by itself \u2014 your handlers still have to be correct under retry.<\/p>\n<h2>Jobs it is good at<\/h2>\n<ul>\n<li><strong>Buffering:<\/strong> producer is fast, consumer is slow.<\/li>\n<li><strong>Fan-out:<\/strong> one event, many independent consumers (email, search index, analytics).<\/li>\n<li><strong>Temporal decoupling:<\/strong> the other service is down; the message waits.<\/li>\n<\/ul>\n<h2>Delivery words that matter<\/h2>\n<ul>\n<li><strong>At-most-once:<\/strong> may drop. Almost never what you want for money or inventory.<\/li>\n<li><strong>At-least-once:<\/strong> the contract to design for. SQS standard queues deliver at-least-once. Kafka and RabbitMQ do too when the consumer acks only after the handler succeeds. RabbitMQ auto-ack is at-most-once: a crash after delivery drops the message. Kafka\u2019s default auto-commit is roughly at-least-once for a synchronous handler and can still repeat work. Handlers <strong>must be idempotent<\/strong>.<\/li>\n<li><strong>Exactly-once:<\/strong> real for a Kafka consume-transform-produce transaction. It does not cover the database write or the email in that same handler. Everywhere else, \u201ceffectively once\u201d means at-least-once plus an idempotent write or an inbox.<\/li>\n<\/ul>\n<p><strong>Competing consumers<\/strong> share a queue (scale workers). <strong>Pub\/sub<\/strong> copies to each subscription. Mixing them up is how you process a payment twice or never.<\/p>\n<h2>Outbox<\/h2>\n<p>Do not <code>SaveChanges()<\/code> then <code>Publish()<\/code> on two connections. Commit the entity and an outbox row together; a relay publishes. The inverse (inbox) records \u201cwe already handled message id X.\u201d<\/p>\n<h2>Poison messages<\/h2>\n<p>A handler that always throws will block a partition or fill a retry loop. After N failures, dead-letter the message with the exception. Alert on DLQ depth. Do not delete silently.<\/p>\n<h2>Example<\/h2>\n<p>The order and the outbox row commit together. A relay publishes the row and marks it sent.<\/p>\n<pre><code>BEGIN;\nINSERT INTO orders (id, total) VALUES ($1, $2);\nINSERT INTO outbox (id, type, payload) VALUES ($3, 'OrderPlaced', $4);\nCOMMIT;<\/code><\/pre>\n<h2>What breaks it<\/h2>\n<p>The process can die after the save and before the publish. The order exists and the rest of the system never hears about it. Use the transaction above.<\/p>\n<pre><code>await db.SaveChangesAsync(ct);\nawait bus.PublishAsync(new OrderPlaced(order.Id), ct);<\/code><\/pre>\n<h2>Pitfalls<\/h2>\n<ul>\n<li>Using the broker as a shared database (querying \u201cthe queue\u201d for state).<\/li>\n<li>Huge payloads \u2014 see <a href=\"\/soft\/message-brokers-and-large-messages\/\">large messages<\/a>.<\/li>\n<li>Chatty request\/response over a queue when HTTP would be a timeout you understand.<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>A message broker is a process that accepts messages, stores them (at least briefly), and delivers them to consumers. You [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[3],"tags":[],"class_list":["post-2918","post","type-post","status-publish","format-standard","hentry","category-software-development"],"_links":{"self":[{"href":"https:\/\/contentlabstudy.com\/soft\/wp-json\/wp\/v2\/posts\/2918","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/contentlabstudy.com\/soft\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/contentlabstudy.com\/soft\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/contentlabstudy.com\/soft\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/contentlabstudy.com\/soft\/wp-json\/wp\/v2\/comments?post=2918"}],"version-history":[{"count":4,"href":"https:\/\/contentlabstudy.com\/soft\/wp-json\/wp\/v2\/posts\/2918\/revisions"}],"predecessor-version":[{"id":3227,"href":"https:\/\/contentlabstudy.com\/soft\/wp-json\/wp\/v2\/posts\/2918\/revisions\/3227"}],"wp:attachment":[{"href":"https:\/\/contentlabstudy.com\/soft\/wp-json\/wp\/v2\/media?parent=2918"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/contentlabstudy.com\/soft\/wp-json\/wp\/v2\/categories?post=2918"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/contentlabstudy.com\/soft\/wp-json\/wp\/v2\/tags?post=2918"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}