Blast from the Past: I filed a bug against Firefox 3.6.6

A screenshot of the old bugzilla duplicate finder UI with the text inside table cells not rendering at allOn June 30, 2010 I was:

  • Sleepy. My daughter had just been born a few months prior and was spending her time pooping, crying, and not sleeping (as babies do).
  • Busy. I was working at Research in Motion (it would be three years before it would be renamed BlackBerry) on the BlackBerry Browser for BlackBerry 6.0. It was a big shift for us since that release was the first one using WebKit instead of the in-house “mango” rendering engine written in BlackBerry-mobile-dialect Java.
  • Keen. Apparently I was filing a bug against Firefox 3.6.6?!

Yeah. I had completely forgotten about this. Apparently while reading my RSS feeds in Google Reader (that doesn’t make me old, does it?) taking in news from Dragonmount about the Wheel of Time (so I guess I’ve always been a nerd, then) the text would sometimes just fail to render. I even caught it happening on the old Bugzilla “possible duplicate finder” UI (see above).

The only reason I was reminded this exists was because I received bugmail on my personal email address when someone accidentally added and removed themselves from the Cc list.

Pretty sure this bug, being no longer reproducible, still in UNCONFIRMED state, and filed against a pre-rapid-release version Firefox is something I should close. Yeah, I’ll just go and do that.

:chutten

 

Advertisements

Data Science is Hard: Counting Users

Screenshot_2018-08-29 User Activity Firefox Public Data Report

Counting is harder than you think. No, really!

Intuitively, as you look around you, you think this can’t be true. If you see a parking lot you can count the cars, right?

But do cars that have left the parking lot count? What about cars driving through it without stopping? What about cars driving through looking for a space? (And can you tell the difference between those two kinds from a distance?)

These cars all count if you’re interested in usage. It’s all well and good to know the number of cars using your parking lot right now… but is it lower on weekends? Holidays? Are you measuring on a rainy day when fewer people take bicycles, or in the Summer when more people are on vacation? Do you need better signs or more amenities to get more drivers to stop? Are you going to have expand capacity this year, or next?

Yesterday we released the Firefox Public Data Report. Go take a look! It is the culmination of months of work of many mozillians (not me, I only contributed some early bug reports). In it you can find out how many users Firefox has, the most popular addons, and how quickly Firefox users update to the latest version. And you can choose whether to look at how these plots look for the worldwide user base or for one of the top ten (by number of Firefox users) countries individually.

It’s really cool.

The first two plots are a little strange, though. They count the number of Firefox users over time… and they don’t agree. They don’t even come close!

For the week including August 17, 2018 the Yearly Active User (YAU) count is 861884770 (or about 862M)… but the Monthly Active User (MAU) count is 256092920 (or about 256M)!

That’s over 600M difference! Which one is right?

Well, they both are.

Returning to our parking lot analogy, MAU is about counting how many cars use the parking lot over a 28-day period. So, starting Feb 1, count cars. If someone you saw earlier returns the next day or after a week, don’t count them again: we only want unique cars. Then, at the end of the 28-day period, that was the MAU for Feb 28. The MAU for Mar 1 (on non-leap-years) is the same thing, but you start counting on Feb 2.

Similarly for YAU, but you count over the past 365 days.

It stands to reason that you’ll see more unique cars over the year than you will over the month: you’ll see visitors, tourists, people using the lot just once, and people who have changed jobs and haven’t been back in four months.

So how many of these 600M who are in the YAU but not in the MAU are gone forever? How many are coming back? We don’t know.

Well, we don’t know _precisely_.

We’ve been at the browser game for long enough to see patterns in the data. We’re in the Summer slump for MAU numbers, and we have a model for how much higher the numbers are likely to be come October. We have surveyed people of varied backgrounds and have some ideas of why people change browsers to or away from Firefox.

We have the no-longer users, the lapsed users, the lost-and-regained users, the tried-us-once users, the non-human users, … we have categories and rough proportions on what we think we know about our population, and how that influences how we can better make the internet better for them.

Ultimately, to me, it doesn’t matter too much. I work on Firefox, a product that hundreds of millions of people use. How many hundreds of millions doesn’t matter: we’re above the threshold that makes me feel like I’m making the world better.

(( Well… I say that, but it is actually my job to understand the mechanisms behind these  numbers and why they can’t be exact, so I do have a bit of a vested interest. And there are a myriad of technological and behavioural considerations to account for in code and in documentation and in analysis which makes it an interesting job. But, you know. Hundreds of millions is precise enough for my job satisfaction index. ))

But once again we reach the inescapable return to the central thesis. Counting is harder than you think: one of the leading candidates for the Data Team’s motto. (Others include “Well, it depends.” and “¯\_(ツ)_/¯”). And now we’re counting in the open, so you get to experience its difficulty firsthand. Go have another look.

:chutten

 

The Photonization of about:telemetry

This summer I mentored :flyingrub for a Google Summer of Code project to redesign about:telemetry. You can read his Project Submission Document here.

Background

Google Summer of Code is a program funded by Google to pay students worldwide to contribute in meaningful ways to open source projects.

about:telemetry is a piece of Firefox’s UI that allows users to inspect the anonymous usage data we collect to improve Firefox. For instance, we look at the maximum number of tabs our users have open during a session (someone or several someones have more than one thousand tabs open!). If you open up a tab in Firefox and type in about:telemetry (then press Enter), you’ll see the interface we provide for users to examine their own data.

Mozilla is committed to putting users in control of their data. about:telemetry is a part of that.

Then

When :flyingrub started work on about:telemetry, it looked like this (Firefox 55):

oldAboutTelemetry

It was… functional. Mostly it was intended to be used by developers to ensure that data collection changes to Firefox actually changed the data that was collected. It didn’t look like part of Firefox. It didn’t look like any other about: page (browse to about:about to see a list of about: pages). It didn’t look like much of anything.

Now

After a few months of polishing and tweaking and input from UX, it looks like this (Firefox Nightly 57):

newAboutTelemetry

Well that’s different, isn’t it?

It has been redesigned to follow the Photon Design System so that it matches how Firefox 57 looks. It has been reorganized into more functional groups, has a new top-level search, and dozens of small tweaks to usability and visibility so you can see more of your data at once and get to it faster.

newAboutTelemetry-histograms.png

Soon

Just because Google Summer of Code is done doesn’t mean about:telemetry is done. Work on about:telemetry continues… and if you know some HTML, CSS, and JavaScript you can help out! Just pick a bug from the “Depends on” list here, and post a comment asking if you can help out. We’ll be right with you to help get you started. (Though you may wish to read this first, since it is more comprehensive than this blog post.)

Even if you can’t or don’t want to help out, you can take sneak a peek at the new design by downloading and using Firefox Nightly. It is blazing fast with a slick new design and comes with excellent new features to help be your agent on the Web.

We expect :flyingrub will continue to contribute to Firefox (as his studies allow, of course. He is a student, and his studies should be first priority now that GSoC is done), and we thank him very much for all of his good work this Summer.

:chutten

Who needs the NSA? FLOSS is bad enough.

Here’s a half-hour talk (with 15min of Q&A at the end) from FOSDEM 2014, an open-source software conference held annually in Brussels. But I know you won’t watch it, so here’s the summary.

NSA operation ORCHESTRA has had a successful first year of operation. With a budget of $1B, and breaking no laws (or even using FISA courts or other questionable methods), they have successfully:

  • kept the majority of Internet traffic unencrypted, so that it can be read by anyone
  • centralized previously-decentralized technologies (e.g. Skype) under companies friendly to the NSA (e.g. Microsoft)
  • created a system of finances that encourages outside influences on proprietary software and transparent bribery
  • created a system of patent expectations that allows companies friendly to the NSA to squelch smaller players that might create disruptive technologies
  • encouraged poor community policies and practices in Open Source projects to help derail useful work through arguments about licenses, naming conventions, and bikeshedding

The joke is that ORCHESTRA doesn’t exist (as far as we know). The software community has been doing this by itself, without encouragement or aid, for years.

“kept the majority of Internet traffic unencrypted, so that it can be read by anyone”

Internet traffic is unencrypted largely because it is expensive to encrypt things. Well, this isn’t true, as one could always self-sign a certificate for one’s webserver and then be able to guarantee that the number of parties in the session does not change in the middle of your transaction. This doesn’t happen because a browser connecting to unencrypted sites displays no warning. But a browser connecting to a site with a self-signed certificate shows a big nastygram that is difficult to understand, let alone bypass:connectionIsUntrusted

Emails are unencrypted because using encryption software on email is difficult, not interoperable, and ceases to work as soon as you want to send email to someone who isn’t capable of decrypting your message. In short: if Amazon can’t encrypt their shipping notification when they send it to you, then it won’t work.

Generic traffic is unencrypted or badly-encrypted (which is, in some cases, worse) because encryption is hard, and the premier tool for using it, OpenSSL, is so poorly documented and has such awful default settings that you’re as likely to encrypt your cat as you are your chat.

(This is where Mozilla could step up with its Mozilla Open Source Support program and offer cash incentives to improve the software and documentation and fund needed audits of the codebase.)

“centralized previously-decentralized technologies (e.g. Skype) under companies friendly to the NSA (e.g. Microsoft)”

Skype, once bought by Microsoft, centralized its encrypted communications under Microsoft’s servers. This aided in the directory service and NAT-busting parts of the protocol, sure, but also provided a single target for subpoenas and other court orders.

As Apple is helping make the public aware, it is beginning to seem as though the safest course is to code things so that even the dev can’t read them.

created a system of finances that encourages outside influences on proprietary software and transparent bribery”

The entire system of Venture Capitalism is rife with abuse and corruption. That funding isn’t coming from the NSA is incidental as VCs come in and drop $10k on a startup in exchange for influence and a piece of the pie. If your funding is coming from someone who thinks you should pivot away from usable crypto, you aren’t going to argue with the paycheque.

Also, if you are the NSA and need an easy way to pay an informant, you can say to her “Tell your boss you quit to form a startup. We’ll set you up in the Valley with $1M in investment. Just surf the web for a year or two.” Easy rewards, perfectly legal.

“created a system of patent expectations that allows companies friendly to the NSA to squelch smaller players that might create disruptive technologies”

Software patents in the United States are an easy tool for larger companies to prey on smaller ones. If you are a small company that patents something that amazingly doesn’t infringe on other patents, a large company, friendly to the NSA, can scoop them up and squelch the startup in a fit of managerial pique. If the patent does infringe on a friend-of-the-NSA’s patents, the small company is squished under the lawyers’ boots.

“encouraged poor community policies and practices in Open Source projects to help derail useful work through arguments about licenses, naming conventions, and bikeshedding”

Here’s the kicker for Mozilla and others trying to make a difference in the Open Source space.

The NSA doesn’t need agents provocateurs to derail conversations. Well-meaning contributors do all the time.

The GCHQ don’t need to distract us with fights about licensing or superficial changes. Users do that all the time.

CSIS agents aren’t sowing doubt about what is better to stop us from doing what is good. Our own internal voices shout quite loudly enough.

…so, what do we do? Well, a good start for those of us in Open Source communities, is to think before we type. Bikeshedding slows us down, but software quality is an imperative. Licensing just doesn’t matter as much as you think it does and is as personal a choice as one’s toenail polish colour.

The next step is to realize that these are generally not technological problems. They are political. They are social. They are human. Developers and technical managers and architects might be some of the best people to understand these problems, but not the best people to solve them.

We need outside help. We need politicians who agree to be helped to understand these complex issues and take action. We need community managers who understand the needs of the project and the needs of humans and how to step between or step away. We need each individual to actively and sincerely stop sucking at talking to others because the harm we’re doing reaches beyond the mailing list and affects users.

We have seen the enemy, and it is us. It is also them. But we can fix “us” so that we are able to fight “them”. I think.

:chutten