Still looking for a sponsor Max Paulousky is looking for a Silverlight/.Net job in the Commonwealth

Share to Facebook Tweet this! Share to MySpace Share to Google Share to Live   Share via AddThis

Problems With Sharing Windows Phone 7 Applications Within A Large Group Of Beta Testers

Abstract

I work for a huge media company in a mobility team as a project coordinator. At the moment, I work on an application for Blackberry and have established workflow for developing and testing new builds. The team includes great number of product owners who work as testers and as customers at the same time. The Blackberry platform provides all required possibilities to have tested applications by any number of testers/product owners. You just need to compile COD files and sign them at RIM to be able to install on any Blackberry device/simulator.

Let’s have a look at a QA cycle for the Windows Phone platform. You can use either a WP7 emulator, that does not support many features like microphone, geolocation, accelerometers etc, or a WP7 device, that should be registered as a developer device ($99 USD per year for three devices).

From my point of view, the current workflow for WP7 is a kind of expensive if you have large QA team. A company should buy developer licenses for QA specialists, developers and for product owners. For example, for the application I am working now it could cost about $1000 USD or even more per year.

I think, Microsoft should propose a new, less expensive way for developing and testing applications for the WP7 platform.

In that article I would like to review several possible ways out.

Way 0. Current way

A company should buy additional developer license for every 3 device. Additionally, testers should somehow install XAP files on their devices.

That way is not convenient but works.

Way 1. Download XAP from web and install

Microsoft can start providing a new license to download and install application on WP7 devices. Developers buy a special license to build and sign XAP files (applications). Customers are able to download (via web link) such XAP files and install on a device as usual applications (because apps are signed with a special license).

This approach is very clear and easy for customers but can cause a lot of issues and risks:

  • Microsoft can’t control who uses an application. I mean 10 testers or ten thousand users will use that application;
  • Microsoft can’t control whether developers charge customers or not (MS can’t get their 30%);
  • This approach is a backdoor that can be used by criminals;
  • This approach requires changes on client devices to allow install apps from web.

Of course, MS can provide additional limitations to improve security of that approach but I don’t think that’s the right way.

Way 2. Buy additional developer licenses for less

MS can change price of annual subscription depending on how many devices can be activated using that license (more devices – less price per device). Well, that’s probably a good solution but that way has the same disadvantages as way #0.

  • QA specialists need to install developer tools to activate their devices as developer ones;
  • There should be created a automation workflow to deliver XAP files on QA’s and customers’ devices.

I don’t think this way is easy to use but definitely more secure then previous one.

Way 3. Using Marketplace

I suppose that the most reliable way to share Windows Phone 7 applications is Marketplace. Everyone, who does not activate own device as a developer one, should install applications via Marketplace. That way has following advantages:

  • Does not require any changes in windows phone runtime;
  • Microsoft can control way of distributing applications

But the current Marketplace should be changed to support big QA teams. In my opinion, following changes should be implemented:

  • Marketplace should support closed groups of users;
  • Development team should be able to add testers/product owners to that closed group (by Live ID);
  • Development team should be able to publish their applications within closed group;
  • Price of a license for publishing applications within closed group should depend on number of group members;
  • Zune should support closed groups.
  • Applications that are published within closed groups, should be available in Marketplace and runable for limited period (e.g. two months) only. On the expiry, MS removes particular version from Marketplace and a device should prevent launching expired applications (it seems, it can be done in various ways);
  • MS do not need to certify applications for closed groups and do not need to restrict number of publishing attempts.

So, the full workflow will look like:

  • Developers publish an application within a closed group on Marketplace;
  • Testers go to a closed group at Marketplace and install/update the application as usual;
  • Developers publish a new version of application on Marketplace;
  • In two months (for example), the application stops launching on a device because it is expired and it disappears from the closed group on Marketplace;

Conclusion

In my opinion, Microsoft needs to introduce some changes in their workflow for WP7 development. Companies with great number of testers do not have convenient way to share beta versions within their testers. I propose several ways out that can be implemented by Microsoft to improve the current workflow.

Dear colleagues!

On the one hand, I welcome your new proposals and ideas to improve and simplify QA workflow to find the best way to test beta versions. On the other hand I would like to hear Microsoft representatives’ thoughts about the problem I raises in this article. Please, do comment my article.

I am sure we will make that process easier and cheaper for everyone.

Update 1: @cliffsimpkins: totally understand and hear you; the QA/beta distro point is something the team is definitely aware of and looking at

This work is licensed under a Creative Commons Attribution By license.

Leave a Comment [ RSS ]

  • re: Problems With Sharing Windows Phone 7 Applications Within A Large Group Of Beta Testers

    Requesting Gravatar... Matt Lacey says:

    Microsoft have repeatedly said they are aware of the demand for this and are working on a solution. It's just not ready yet. (They have hinted at dates previously but these have passed with nothing to show for it.)

    Just have to be patient and see what the next few months bring....

  • re: Problems With Sharing Windows Phone 7 Applications Within A Large Group Of Beta Testers

    Do agree, Matt
    Cliff's (WP7 PM at MSFT) sentences and question to me in Twitter confirm that they are working on

  • re: Problems With Sharing Windows Phone 7 Applications Within A Large Group Of Beta Testers

    Should be some SAAS or something like that to resolve it, however if M$ allow it

  • re: Problems With Sharing Windows Phone 7 Applications Within A Large Group Of Beta Testers

    Oleh, MSFT has all required infrastructure at the moment. They need just extend it, not implement all allow anything else and they work in the right direction

  • modern sofa

    Requesting Gravatar... modern sofa says:

    Hello
    PROBLEMS WITH SHARING WINDOWS PHONE 7 APPLICATIONS WITHIN A LARGE GROUP OF BETA TESTERS
    Fairly excellent submit, This webpage presents helpful data to us, I praise you
    Thank you!

Comments have been closed on this topic.