Distributed Teams: A Test Failing Because It’s Run West of Newfoundland and Labrador

(( Not quite 500 mile email-level of nonsense, but might be the closest I get. ))

A test was failing.

Not really unusual, that. Tests fail all the time. It’s how we know they’re good tests: protecting us developers from ourselves.

But this one was failing unusually. Y’see, it was failing on my machine.

(Yes, har har, it is a common-enough occurrence given my obvious lack of quality as a developer how did you guess.)

The unusual part was that it was failing only for me… and I hadn’t even touched anything yet. It wasn’t failing on our test infrastructure “try”, and it wasn’t failing on the machine of :raphael, the fellow responsible for the integration test harness itself. We were building Firefox the same way, running telemetry-tests-client the same way… but I was getting different results.

I fairly quickly locked down the problem to be an extra “main” ping with reason “environment-change” being sent during the initial phases of the test. By dumping some logging into Firefox, rebuilding it, and then routing its logs to console with --gecko-log "-" I learned that we were sending a ping because a monitored user preference had been changed: browser.search.region.

When Firefox starts up the first time, it doesn’t know where it is. And it needs to know where it is to properly make a first guess at what language you want and what search engines would work best. Google’s results are pretty bad in Canada unless you use “google.ca”, after all.

But while Firefox doesn’t know where it is, it does know is what timezone it’s in from the settings in your OS’s clock. On top of that it knows what language your OS is set to. So we make a first guess at which search region we’re in based on whether or not the timezone overlaps a US timezone and if your OS’ locale is `en-US` (United States English).

What this fails to take into account is that United States English is the “default” locale reported by many OSes even if you aren’t in the US. And how even if you are in a timezone that overlaps with the US, you might not be there.

So to account for that, Mozilla operates a location service to double-check that the search region is appropriately set. This takes a little time to get back with the correct information, if it gets back to us at all. So if you happen to be in a US-overlapping timezone with an English-language OS Firefox assumes you’re in the US. Then if the location service request gets back with something that isn’t “US”, browser.search.region has to be updated.

And when it updates, it changes the Telemetry Environment.

And when the Environment changes, we send a “main” ping.

And when we send a “main” ping, the test breaks.

…all because my timezone overlaps the OS and my language is “Default” English.

I feel a bit persecuted, but this shows another strength of Distributed Teams. No one else on my team would be able to find this failure. They’re in Germany, Italy, and the US. None of them have that combination of “Not in the US, but in a US timezone” needed to manifest the bug.

So on one hand this sucks. I’m going to have to find a way around this.

But on the other hand… I feel like my Canadianness is a bit of a bug-finding superpower. I’m no Brok Windsor or Captain Canuck, but I can get the job done in a way no one else on my team can.

Not too bad, eh?

:chutten

Advertisements

Canadian Holiday Inbound! Thanksgiving 2018 (Monday, October 8)

Monday is Thanksgiving in Canada[1], so please excuse your Canadian colleagues for not being in the office.

We’ll likely be spending the day wondering. We’ll be wondering how family could make such a mess, wondering why we ate so much pie, wondering if it’s okay to eat turkey for breakfast, wondering if pie can be a meal and dessert at the same time, wondering how we fit the leftovers in the fridge, wondering why we bothered hosting this year, wondering whose sock that is by the stairs, wondering when the snow will melt[2] or start to fall[3].

We’ll also be wondering who started the family tradition of having cornbread instead of buttered rolls, wondering where the harvest tradition began, wondering about what all goes into harvesting our food, wondering what it means to be thankful, wondering what we are thankful for, wondering why we ate the evening meal at 4pm, wondering whether 4pm is too late to have a nap.

With heads full of wondering and bellies full of food, we wish you a wonderful Thanksgiving. We’ll be back to work, if not our normal shapes, on Tuesday.

:chutten

PS: Canadian Pro-tip: Leftover food often turns into regret – but this regret can turn back into food if you leave it in the fridge for a little while!

[1]: https://mana.mozilla.org/wiki/display/PR/Holidays%3A+Canada
[2]: Calgary had a (record) snowfall of 32.8cm (1’1″) on Oct 2: https://www.cbc.ca/news/canada/calgary/calgary-october-snow-day-two-1.4848394
[3]: Snow’s a-coming, already or eventually: https://weather.gc.ca/canada_e.html

The Mystery of the Lloydminster McDonald’s

On our way back from a lovely family afternoon out at the local streetcar museum, we hit up a McDonald’s for a quick bit of dinner on our way home. There we found a new promotion:IMG_20171209_185212.jpg

So far so blah.

But then I looked closer:

img_20171209_185212-e1515009114286.jpg

I’m upgrading my phone shortly, so hopefully you won’t have to suffer such poor-quality images in the future. But for now, allow me to transcribe:

“Not valid with any other offer. At participating McDonald’s restaurants in Ontario, Quebec, Atlantic Canada, Alberta, Northwest Territories and Lloydminster, SK.”

First: What is Atlantic Canada? I know it’s a colloquial designation for the four Eastern provinces (New Brunswick, Prince Edward Island, Nova Scotia, and Newfoundland and Labrador), but I didn’t expect to see it referenced in legalese at the bottom of promotional copy. Apparently the term is semi-legitimate, as there is an official arm of the federal government called the Atlantic Canada Opportunities Agency/Agence de promotion économique du Canada atlantique. Neat.

Second: Oh good, Quebec gets the promotion, too. Quebec gets left out of many things advertised to the whole of Canada because of stricter laws governing gambling (including sweepstakes. Tim Hortons has to bend over backwards to make Roll Up the Rim To Win work there) and advertising (especially to children).

Third: What did Manitoba, Saskatchewan, British Columbia, and the other two territories (Yukon and Nunavut) do to be left out? Nunavut seems obvious: there are no McDonald’s restaurants there. But there’s at least two golden arches in Whitehorse, and scads in Manitoba, Saskatchewan, and BC. Must be some sort of legal something.

And that brings us to the titular mystery: Why, of all of Saskatchewan, is Lloydminster spared? Is it the one city where there’s competition? Is it to do with that urban legend about an older burger restaurant in Western Canada someplace that was called McDonald’s first? Is it because it’s licensed under a special food services employer contract that….

No.

It’s because Lloydminster Saskatchewan is here:

lloydminster

It straddles the border between the provinces Alberta and Saskatchewan. Alberta has the McDonald’s promotion. Saskatachewan doesn’t. Lloydminster, AB has two McDonald’s restaurants. Lloydminster, SK has one. It would be unfair to deny the Albertan restaurants the promotion, and unfair to exclude the Saskatechewanian restaurant. So what is McDonald’s to do?

They have to add a rider on their promotion to all corners of the Great White North that proclaims that there is one city (barely) in Saskatchewan in which you can partake of a five dollar meal deal.

Yeesh.

Canadian Holiday Inbound! (Sunday December 25th, Observed on Monday the 26)

Hello team!

A Canadian Holiday is once again at our chimneys as Christmas Day is approaching! The holiday itself is on the 25th, but because that falls on a weekend, it’ll be on the 26th that you’ll not be finding us in the office. Please be understanding if your meetings are a little underpopulated, and take the opportunity to run all the try builds you can think of.

Now, I know what you’re thinking. (Well, I don’t, but since I’ve turned off comments you can’t correct me.) You are thinking that Christmas isn’t a Canadian holiday… or at least not uniquely Canadian.

And you wouldn’t be too wrong. Other parts of the world certainly celebrate Christmas. Japan does it up amazingly, for instance, if you’re ever in that corner of the globe in December. And still other parts of the world people celebrate all sorts of Winter festivities.

And it’s not as though we’re spending the day dipping our All-Dressed Chips[1] into Stanley Cups[2] of Maple Syrup[3] while taking our Coffee Crisp[4] and wearing our Toques[5], Jeans, Jean Shirts[6], and A Boot[7], eh?

No, but the Canadian way of celebrating Christmas _is_ unique… in that it’s usually done through celebrating everyone else’s Winter celebrations. Canadians are more than happy to adopt and support any culture or festival that involves food, fun, friends, and family.

My family tends to observe Polish Wigilia by eating pierogies, white fish, and bigos. My wife’s family has a Christmas Eve Feast of crab dip, fourteen types of frozen hors d’oeurves, cheese, crackers, and smoked oysters eaten on TV trays in front of Log: The Christmas Special. Earlier this month we ate latkes with sour cream and applesauce with pfeffernusse for afters at the Christkindle Markt. Last year we went to Sir John A. MacDonald’s birthplace at Woodside for soft gingerbread and roasted chestnuts.

Then there’s turkey with the trimmings for the more traditional, sushi for deliberate anti-traditional, and everything in-between.

So no matter if or how you celebrate Canadian Christmas, know that we are (and are not) celebrating it too, with you, in the Great White North.

Because anything else just wouldn’t be polite.

( :bwinton reminds me to tell you that we will also be off on the 27th for Boxing Day. Our most famous pugilists will be hard at work discouraging (in effigy) the normally-docile moose herds from invading the United States once again. So we’ll be busy cheering them on, sorry. )

:chutten

[1] Tastes like… actually, I’m not really sure. Tasty, though.
[2] Named after Lord Stanley
[3] Probably harvested back in March in Quebec
[4] mocha-flavoured Nestle chocolate bar
[5] knit caps, often with pom-poms on top
[6] AKA “The Canadian Tuxedo”
[7] “about” pronounced in Canadianese is actually closer to “aboat” than “aboot”, eh