I’ve been reviewing iPhone apps, for almost a year now, and over the past, I have added new elements to how I review the apps, like ratings, and etc.

I have been reading my past reviews, and now I feel like I have not been giving really good and thoughtful review of the apps. And the rating system I have on the app, has been, let’s just say, sloppy.

This problem really took notice after reviewing the app, MiMusic. I thought I did not give thoughtful review of the app.

So, over the last couple of days, if not months, I have been planning a complete overhaul of the app review process, and today, I would like to share with you the final details of the new app review process.

First of, the ratings. I have been giving an overall number on the app, out of 10, with 10 being the best. Now, I have structured rating system based on five categories: Compatibility, Interface, Direction and Ease of Use, Price, and Performance.

1. 1.Compatibility with All iPhone OS devices that Apple supports: (out of 6 points)
   

2. •iPod touch (1st generation) (1 point)
   

3. •iPod touch (2nd Generation) (1 point)
   

4. •iPod touch (3rd Generation) (1 point)
   

5. •iPhone (1st Generation) (1 point)
   

6. •iPhone 3G (1 point)
   

7. •iPhone 3GS (1 point)
   

I have noticed that some apps I have are fully capable on running on devices older iPhone and iPod touch without any problem. I know some people would say that new apps need newer devices, due to hardware limitation, but I know, some apps do not need special new hardware introduced in newer devices. Therefore, I have added the compatibility to the things I am looking for.

However, if the app is specifically tailored for specific application within the default iPhone OS application, or limited by hardware limitation (such as interaction with phone application, or magnetic digital compass on iPhone 3GS), I will put N/A on the specific devices, making the compatibility section either out of 3, or 2, or 1.

There are exception to the above rule, which is Video recording. Since Apple now allows video recording application on App Store, all the iPhone models are now capable of recording video. Therefore, if the video recording application is only tailored toward 3GS, then it will automatically be rated 1/3, since hardware is no longer a limitation.

2. 2. Interface (out of 7 points, 6 if transition is not included, 5 if no sound is included, 4 if special instructions are needed to use the application)
   

3. • Clean and Concise (out of 3 points, with 1 being the worst)
   

4. • Easy to understand, RIGHT OFF THE BAT without any direction (If applicable)
   

5. • Transition animation is smooth, and not jagged (if applicable)
   

6. • Accessible to people with disabilities
   

7. • Sound is smooth, and matching to each action
   

The interface portion of the review process is plain and simple. The clean and concise portion of the process is worth 3 points, since clean interface is the most important portion of the app. I am sick of seeing sloppy interfaces on app, and I hope this will wake the developers up that clean interface is critical in the review process.

I also put accessibility. I plan to test some of the apps with screen readers on iPhone 3GS. Accessibility is the most important thing in the app store. Consider this portion as “does the app pass the ADA rules?”

Also, I added the fact that apps need to be so easy to use, that users do not need direction to find out how apps work. The only exception to this rule is apps that is on specific subject, and that apps need special instruction before use, or games.

I also added animation and sounds. If the transition animation looks jagged, or slow, it will be marked down. Also, transition sound must match with the appropriate action within the app.

3. 3. Directions (out of 2 points, 4 points if it’s a game)
   

4. • Apps have directions I can look back to if I forget how functions work (2 points, 4 points for game, one for direction and one for game rules)
   

5. • Apps have directions that are easy to understand, and users should be able to understand without having to look back to the direction frequently (2 points, 4 points for game, one for direction, and one for game rules)
   

Directions are ABSOLUTELY Critical when it comes to any apps, ESPECIALLY GAMES! That’s why for this category, each criteria is worth 2 points for gaming apps. This is because in gaming, rules also apply. For all the other apps, it’s only direction, so each criteria is worth 1 points.

For example if it’s gaming app, for the option to look back into the direction, I will check if I can go back to check the rule of the game, (as one point), and also the direction of how to play the game (as one point). This makes it two points so far if both criteria is met. I then check if I can understand each rule (as one point) and each directions (as one point) easily, without having to go back. If those criteria is met, it’s total of four points for the directions category.

4. 4. Pricing of the apps. (depends on the functions of the app, and also the size of the app, applies to full version of the app, not lite version. Highest point is 50)
   

5. • Overpriced (-15 points)
   

6. • Nickel and Diming “aka, the app that is free, but you have to buy something to really add more features. Excludes games, and service apps” (20 to -10 points, depending on how useful the application is worth without the in app  purchase)
   

7. • Somewhat overpriced, but still affordable, but still requires in app purchase to add full content to the app (-10 points)
   

8. • Price raise without notification (-5 to -30 points, depending on how much the developers raise the price to)
   

9. • Somewhat overpriced, but still affordable, no in app purchase needed (10 points)
   

10. • Affordable, in app purchase needed to add full function of the app (10 points)
    

11. • Affordable, no in app purchase needed (20 points)
    

12. • 99 cent bargain! (30 points, with or without in app purchase)
    

13. • FREE FOR ALL! (30 points, no in app purchase needed)
    

I strongly believe that all apps on the app store should be affordable to customers the developers are tailoring to. This is why I introduced this category.

If you have not noticed, this is one of the category where you can lose points. YES, YOU CAN LOSE POINTS IN THIS CATEGORY. I specifically did this, because WE ARE IN RECESSION. People are trying to save money in any way possible, so we can survive in this turmoil, and this means saving money on apps. In fact, research shows, that saving money by getting apps for free really saves about $100 bucks in the long run. This is why the pricing category is huge!

5. 5. Performance (Out of 10 points)
   

6. • Amount of times application crashes, based on every day use (0-3 points, 0 for having most crashes in everyday use)
   

7. • Application does not crash based on low memory issue (-10 points)
   

8. • Application has smooth animation throughout, no jagged animation, or rough animation (if applicable, 3 points)
   

9. • Application does not hang, or slow down due to auto-email pushing, incoming text messages, or push notification “aka, the everyday use of iPhone setting” (0-4 points, 0 having huge hang and slowdown, and ultimately crashing)
   

Even with Apple’s strict App Store approval process, some apps ends up having bugs, and etc, and this is due to lack of complete and through testing. This is why I am armed with instrument monitoring application, to make sure the app does not behave abnormally. Because the last thing any users want is an application that crashes during the critical moments.

And if you have noticed, this is another category where I deduct points.

These five categories are what I call, “the basics of iPhone application”. These five categories are what developers need to think when it comes to development. This is why I based the rating on these five categories.

In the end, I will total up the points, and then divide the points by the maximum total number of points the developers would have gotten to calculate the percentage. 100% being the Platinum Star app, 99 ~ 90% being the Orange Metallic star, and 89 ~ 85% being the Black Metallic star.

Not only will I review the app based on these five categories, but I also plan to re-visit some apps at later time, and will do the whole process all over. But in addition to the ratings above, I also plan on giving additional feedbacks on the following:

1. • Developer’s behavior: I believe this is controversial one, but I must stress, customer relations is important in the app industry. If you are being an jackass to customers and all you care about is money, then I will give penalty points from the total before percentage calculation. I will check on this using a unique way, which I am not saying. (I do not want developers to cheat)
   

2. • Price Raise / Price Cuts: One thing that I hate the most is the price changes in the app. Especially when the paid app becomes free, and free app becomes paid, or paid app becoming ridiculously expensive. Just like the developer behavior, I will also give penalty points on this as well, depending on how bad the price change is.
   

Big changes, don’t you think? This is why I am announcing right now,

THAT ALL APP REVIEWS ALREADY POSTED WILL BE RE-REVIEWED UNDER NEW METHOD AS BRAND NEW APPS!

Get ready developers, for real honest feedbacks from me.