The Hidden Bill Shock of Bengali Text Messaging
Many marketing teams and developers in Bangladesh are shocked when an SMS campaign invoices at double or triple the anticipated cost. The culprit is almost always character encoding mismatch: an accidental zero-width space, curly quote, or a single Bengali character converting an entire 160-character message into expensive 70-character UCS-2 chunks.
1. The Technical Architecture: GSM 03.38 vs UCS-2 Encoding
Telecommunication protocols allocate a maximum payload of 140 octets (1,120 bits) per SMS User Data Header (UDH). How that storage is partitioned depends strictly on the character set:
| Encoding Standard | Bit Depth | Single SMS Limit | Concatenated Part Limit | Applicability |
|---|---|---|---|---|
| GSM 03.38 (Default) | 7 bits / char | 160 Characters | 153 Characters / part | Standard English A-Z, digits, common symbols |
| UCS-2 (Unicode) | 16 bits / char | 70 Characters | 67 Characters / part | Bengali (বাংলা), Arabic, Emojis, Smart quotes |
2. Why Concatenation Eats Your SMS Budget
When a message exceeds the single SMS threshold, telecommunication operators split the text into concatenated segments and attach a 6-byte User Data Header (UDH) so the receiving phone can reassemble them.
For GSM text, the per-part limit drops from 160 to 153 characters. But for Bengali Unicode, each part drops from 70 to just 67 characters. A 140-character Bengali promotional message is billed as 3 SMS parts, whereas the English equivalent costs only 1 SMS part!
3. Sneaky Invisible Characters that Trigger UCS-2 Conversion
Even when writing pure English text, developers often encounter unexpected UCS-2 billing due to:
- Curly "Smart" Quotes: Copy-pasting from Microsoft Word or Google Docs replaces ASCII straight quotes (
",') with typographic curly quotes (“,”,’), triggering instant 70-character conversion. - Unicode Em-dashes: Dashes like
—instead of standard hyphen-. - Zero-Width Joiners (ZWJ): Invisible characters introduced when typing complex Bengali conjuncts (যুক্তাক্ষর) like
ক + ্ + ষ. - Trailing Emojis: A single 🎉 or 👍 at the end of an English message immediately collapses the character limit to 70.
4. Sanitize and Count: Python & PHP Code Snippets
Before dispatching messages to Cloud SMS BD HTTP APIs, pass your payload through a sanitization function to strip non-GSM characters:
function sanitizeGsmSms($text) {
$gsmRegex = '/^[A-Za-z0-9 \r\n@£$¥èéùìòÇØøÅåΔ_ΦΓΛΩΠΨΣΘΞÆæßÉ!"#%&\'()*+,\-.\/:;<=>?¡ÄÖÑܧ¿äöñüà]*$/';
$isGsm = preg_match($gsmRegex, $text);
return [
'encoding' => $isGsm ? 'GSM_7BIT' : 'UNICODE_UCS2',
'part_size' => $isGsm ? (mb_strlen($text) > 160 ? 153 : 160) : (mb_strlen($text) > 70 ? 67 : 70)
];
}
Test Your Messages Live
Use Cloud SMS BD's real-time interactive character counter on our homepage to preview parts, billing, and operator cost before sending.
Open Live SMS CalculatorAbout Tech Team
API Architecture Lead at Cloud SMS BDManaging telecom SMPP gateways, high-concurrency Redis queues, and REST API microservices for Bangladesh's top fintechs and e-commerce platforms.
