Friday, April 16, 2010

What does it take to invalidate a strategy

The #Wikipedia strategy presented by the WMF director is significantly at odds with my perception of the Wiki world. I have argued in the past against an overly reliance on numbers.

It is easy to show fault at some of the statistics. Those faults can be remedied, but in my opinion allowing for those faults, the arguments that allow for the persistence of those faults are what invalidates the statistics and the conclusions based on them.

screenshot at 18:48 Amsterdam time

It has already been announced that there are a billion edits. This is at odds with the counter that I blogged about in the past and, this counter is generally known. I discussed the higher number with Avatar and he had to admit that even his numbers were wrong.

Numbers and by inference statistics support a point of view. When you use statistical analysis properly you infer from the numbers the reality of a situation and this allows for the formulation of a strategy. All too often the result has been predetermined and consequently the approach to the numbers will not only confirm held convictions, they prevent optimal results.

I wonder if the WMF is willing to consider the bias in its numbers and the impact on its strategy.
Thanks,
      GerardM

6 comments:

Erik Z said...

Gerard, in one post you vent general but unspecified concerns about how WMF uses wrong numbers for their strategy plan, and also how two counters for a one day celebration are less than one day apart, am I right?

If you believe the counter you show a thumbnail of is right, please concentrate on celebrating first. :-)

GerardM said...

One issue with the numbers is that their selection makes them self fulfilling prophecies. There are issues with the validity of some of the numbers and as a consequence I ask the question if the WMF is willing to consider its bias.

The counter is a case in point; the counter was universally accepted but Avatar indicated that the numbers were wrong. His moment of glory was tweeted earlier in the day. I discussed the numbers with Avatar and we agreed that his original number is wrong as well.

The billion edits mark probably passed yesterday. In many ways this spoiled the pleasure of reaching this perfect number.. I had my fun watching the extra number being added anyway :) ..

I have brought my issues with the numbers to the attention on several occasions. In my opinion it adversely affects what we can and will achieve.

The question as I put it allows me to observe the reaction of the WMF. As you know, I want the best for the WMF.
Thanks,
GerardM

Erik Z said...

"There are issues with the validity of some of the numbers"

"I have brought my issues with the numbers to the attention on several occasions."

Gerard, I don't expect many wikimedians have memorized all your blog posts. If you want to make a point you have to do better than that (after a good night sleep). And mingling countdown sloppiness and strategy issues won't help to convince anyone.

Avatar said...

To add some more background information: The graphical counter omits edits from the mediawiki.org domain - my counter included them (~300.000 edits difference).

Like usual this is a question of definition - what exactly should the number show? What is a Wikimedia project? What is "an edit"? How is it counted? etc.

We know this problem in a smaller perpective from milestones in one project like e.g. the 1, 2 or 3 mm edits in English Wikipedia. It's not really possible to specify THE article but the community more or less choose one.

The current problem "identifying the 1 billionst edit" is a lot bigger because you have edits and not articles and you need to examine 700+ projects. That's not really possible.

On the other hand I think you're going to far to state that the strategy plan of the WMF bases on wrong numbers - perhaps because I haven't read all of your blog posts in the past. The current "problem" of identifying the correct billionst edit is of course not really a problem.

GerardM said...

The issue with the billion edit mark is an illustration to the more basic question, what does it take to invalidate a strategy.

The issue is not with taking either number, the issue is with selecting a baseline, having community acceptance of that baseline and abandoning it at the last number.

The billion edit counter is unrelated to the numbers the strategy is based on. My post is related to Sue's presentation, the official statistics. Philipe is the guy who adopted your numbers without checking their validity.
Thanks,
GerardM

Unknown said...

Hey Gerard, can you separate the two issues? If you've got concerns about the data/analysis underpinning the strategy, I'd like to know the specifics.

(Also -- I am generally okay with data that's "good enough" for decision-making, and I don't think imperfect data by definition invalidates the decisions that come out of it. But at this point I don't know what you think is flawed.)

And -- hello from Berlin! I wish you were with us :-)