The confusing state of cloud SMS

kheimerl's picture

We’re in Desa, a small town in Papua, Indonesia, setting up TIER's Village Base Station, a low-power, low-cost GSM network designed for rural areas. While we're also interested in things like building locally-relevant and useful services for the users of our networks, one of the main values of bringing cellular phone service to a place like Desa is connecting people to the outside world. Being able to make a phone call or send an SMS to Wamena, the nearest large town, can save a day's worth of travel and a large amount of money. Understandably, the community is most excited about being able to communicate with the outside world. We're getting asked every day, "When will the network be ready? When can I buy a SIM card?" And our answer for the last few months has been, "As soon as we get everything worked out with our provider."

 

It took us about four days to set up a working GSM system here, complete with local calling and SMS as well as the other phone company things like billing. We even set up outgoing calling with VOIP providers in Indonesia and the US (we initially decided for some boring and involved reasons not to support incoming calls). All that was left to do was set up a way to send and receive SMS to the outside world so our users could contact the outside world. Should be easy, right?

 

Our first stop was Twilio. We’ve used them for hacking up simple telephony apps before, and they’ve worked fine in the past. Even better, they have good rates for sending and receiving SMS in Indonesia, and they explicitly state they support sending SMS to all the major providers in the country (including Telkomsel). Say what you will about Twilio, they’re very easy to work with, and it took us about a day to put together everything we needed with them. With the network built, we got a few locals to SMS their friends from our network. People using the smaller GSM operators in Indonesia (IndoSAT and XL) received and replied to messages just fine, but we didn’t hear any responses from Telkomsel customers, the biggest provider in Indonesia (and primary provider in Wamena). We sleep, assuming that perhaps we were just unlucky with timing.

 

The next day, we come to the realization that the messages were not delivered; our friend in Surabaya would have responded by now. We emailed this friend, confirmed that he didn’t receive any SMS, and asked him to send an SMS. We receive HIS message.That brings us to to the first confusing step of this journey: there are different send and return paths for SMS. This isn’t *that* surprising honestly; these things often happen in the internet world if a routing table is wrong. Maybe there is a misconfiguration somewhere that would cause this. We file a ticket with Twilio, and the next day we get our response. It’s not that there’s a routing issue, it’s that Telkomsel blocks numbers from the +18 prefix... and we happened to have selected numbers from Knoxville, TN (+1865). While this strikes us as bizarre, it seems within the realm of possibility, so we get different numbers from Twilio and try again, with the same result. Outbound is blocked, inbound is not. We file another ticket and decide that maybe we need a backup.

 

Enter Voxeo. We note that we’re good friends with a few people at Voxeo, having built the last network at burning man with Tim Panton and Johnny Diggz. Voxeo provides a Twilio competitor named Tropo which is more expensive than Twilio for Indonesian communications, (3USD vs 1USD per number per month and 2c vs 1c per SMS) but we just need something that works. We try to buy a number, but it’s a lot more complicated than Twilio: they require a justification for our desire to send an international SMS. After jumping through that hoop and figuring out their confusing API, we buy the number and send an SMS to our friend in Surabaya. Now he receives our message but we don’t receive his response. The exact opposite of Twilio. We file a ticket there as well. Their response (20 minutes!) is that US numbers are blocked, we should try a Canadian one. We try it, and this, as you may guess, doesn’t change anything. Next ticket: Their carrier doesn’t support receiving Indonesian SMS. End of Tropo story.

 

Enter Bandwidth.com, another “cloud” provider. They actually ask for an NDA to even begin figuring out how much it will cost to send SMS, but we couldn’t have signed the NDA even if we wanted to since their sales team has yet to respond to our emails. We abandon that plan quickly.

 

So, we’re sitting on our Twilio ticket, which takes around a day for each back-and-forth communication. We wish to reiterate that Twilio *specifically* states that they work with Telkomsel. They go back and forth with their provider, changing routes to no effect. Their provider claims that Telkomsel doesn’t accept ANY USA numbers, which we immediately call bullshit on as our Surabaya friend regularly SMS’s his daughter in Florida. We again decide to pursue other options.

 

Enter SMS Spoofing. There are a number of online services which claim to let you spoof the return address of an SMS. This is illegal in the US, but legal in places like Australia, which is conveniently close to Indonesia. The goal is to spoof our totally legitimate return Twilio number, so that we have two-way communications. We explore this seedy underbelly of cloud telephony but find no services which can spoof onto Telkomsel. Again a dead end.

 

After two weeks of wrangling, engineering, hustling, and finagling, we receive the final word from Twilio: they do not and will not support service to Telkomsel after all. They have removed Telkomsel from their list of providers (http://www.twilio.com/sms/pricing/id, impact!) and that’s that. We’re back where we started. We throw out a wider net, finding Interactive Digital Media (IDM) via a LinkedIn post to an SMS gateway group. They, wonderfully, spend a few nights trying many different providers and numbers demonstrating the following: Telkomsel DOES block US numbers, but primarily from bulk SMS carriers. During this time, Telkomsel actually started sending incorrect return addresses for US ATT numbers, confirming that the problem is more general.

 

With this new knowledge in hand, we are able to search for SMS providers in international areas, arriving at Nexmo (www.nexmo.com). Nexmo is an SMS-only twilio competitor located in the UK with a similar REST API and having cheaper (.05EUR/mo, .009EUR/msg to Indonesia) Swedish and UK numbers. While waiting for IDM to come back from Christmas vacation, we start testing Nexmo (with Dutch and Swedish numbers) and find that it can send and receive SMS from Telkomsel, but the return address is getting mangled. It would literally show up as either a long IndoSAT number or complete gibberish. We create a ticket on their system on the night before new years and almost immediately receive a response: they suggest trying UK numbers and will change the carrier on the back side. At this point, literally new years eve, the system finally has full-duplex SMS. Success, and amazing customer service.

 

Now, this solution isn't optimal; we'd obviously prefer Indonesian numbers. Responding to our SMS costs 650rp (roughly 6.5c) which is much higher than most Indonesian communications. However, after a two-month journey through cloud telephony, we finally have an answer. An, as hope springs eternal, Nexmo thinks they might have a source of Indo numbers.

 

Here are the lessons I think we gathered from this journey: First, don't assume global connectivity. There are a lot of games being played by a lot of stakeholders and these often don't simplify to "connect people". Second, being an SMS gateway is a sketchy business; there are many people using the same systems we are for international spam and scams. Lastly, test the "easy" parts. We spend more than a few days building systems that relied on unreliable external processes like Twilio.

 

I want to thank the customer service staff of Twilio (specifically Dan Markiewicz and Elaine Tsai), who helped a great deal even if it didn't work out. I'd also like to thank David Vigar at Nexmo.com for being incredibly motivated to help us solve our problems. He seriously responded on new years eve and apologized for his connections not responding in kind. Hopefully our network will be driving a lot of traffic through theirs, leading to a mutually beneficial engagement. If I do this again in another country, I'll certainly be using their service.

 

Lastly, here are the usage statistics for the last week of outbound SMS using Nexmo with just 5 active real-world users of the network. Things are looking promising!

 

SMS Usage Statistics