Database Cleanup

Recently, SourceForge updated their site with a new layout, breaking several application entries in the AppSnap database. Since it was quite a bit of work figuring out which apps were broken and how to fix them, I added some functionality within AppSnap to facilitate the app fixing process. After a couple boring hours last evening, I’m happy to say all apps are functioning correctly again. If you find any broken apps, please feel free to let me know.

Categorization Improvements

It took a while but I have finally completed re-categorizing all 333 applications in the AppSnap database into a more sensible scheme. The original categorization was too trivial once the database grew beyond 50 applications. The current scheme is inspired from many existing schemes out there so it was not hard to come up with. It did take quite a bit of time to relabel each app and then migrate the changes to Zoho. Without Zoho Creators batch import facility, this would not be as trivial. Thanks again Zoho

The current GUI does not leverage this schema too well. The new GUI I am working on will do a much better job and make finding applications much easier than before.

As usual, feedback appreciated.

AppSnap TODO published

I finally gone ahead and published the long list of TODO items I have been maintaining for AppSnap. I will be keeping this up to date so it should be an accurate representation of what AppSnap will be doing in the future. This list will complement the issue tracker since it is much easier to maintain as a text blob than having to post and manage a long list of enhancements. Feel free to comment on the list. Any discussions emerging from your feedback will surely go towards improving AppSnap.

I have also changed the WordPress theme for this blog from Contempt to Neat, since it is, in fact, neater. The bullets look so much better now than before.

Not many updates as of yet in terms of features but the database is being constantly expanded and bugs being fixed as often as time permits.

Tweeting about AppSnap

I recently created a Twitter account to provide frequent updates on what is going on as far as programming and technology is concerned. I have discussed this a bit on my personal blog. The account is generic and I plan on using it for all my programming adventures but it will help get an insight on what is going on with AppSnap, in between blog posts and releases.

Feel free to communicate with me over Twitter if you are an active user there.

Adding applications to AppSnap

Adding applications to AppSnap takes longer then you would think. Each application needs to be downloaded, installed and then uninstalled in order to get all the information that AppSnap needs. On an average, it takes a good 10-15 minutes per application, excluding any download time. That’s not too bad when you start out but after a point, it is too tedious to deal with.

With 1.3.3, AppSnap crossed the 200 mark in terms of supported applications. It took almost a year and a half to get that far and that reflects the pains associated with growing the database. Considering I am the only guy adding applications as well as enhancing AppSnap, both on a part-time basis, it makes it that much slower. Something had to change.

It is not possible to automate database additions since each application is unique. However, it is possible to speed up the steps that it takes add an application. To make life a little easier, I added the -a flag to the AppSnap CLI to facilitate adding applications. Using -a, it now takes at most five minutes to get an application added to AppSnap (excluding download time). If you haven’t noticed already, the database now has 285 applications and counting. That is more than 80 applications in 12 days. Not bad.

Adding applications is not only faster, but a lot more fun. Indeed, it is the small things that make a big difference.

AppSnap 1.3.3

I finally got around to releasing version 1.3.3 of AppSnap. It took a lot longer than I expected but I wanted to be sure that the update mechanism was solid. This would ensure that any interim updates will be accessible to users without requiring a new build.

AppSnap 1.3.3 was long overdue considering over 18 bugs have been fixed since 1.3.2 came out. There were many issues being reported by users, most of which were resolved with these fixes. Considering pre-releases do not get as widespread distribution, the fixes were not getting out. With the new update mechanism, such delays will be less likely.

There are also several enhancements worth mention. As usual, the change logs since 1.3.2 list all these improvements. Some of the previous posts also discuss some aspects in detail. In a nutshell, this new version adds the following new features:

• Support for Firefox and Thunderbird add-ons
• Update detection and self-update without reinstallation
• Web-based database management
• Improved version detection and text replacement engine
• Support for multiple databases

Thanks for all the suggestions and bug reports. Have fun using the new version. You can download it from here.

Generic version replacement – II

I have successfully implemented a replacement for #DOTLESS_VERSION#, #DASHTODOT_VERSION#, etc. as mentioned previously. This successfully deprecates the following existing version replacements as follows:-

• #DOTLESS_VERSION# = #REPLACE([.],,#VERSION#)#
• #DASHTODOT_VERSION# = #REPLACE(-,.,#VERSION#)#
• #DOTTOUNDERSCORE_VERSION# = #REPLACE([.],_,#VERSION#)#
• #DOTTODASH_VERSION# = #REPLACE([.],-,#VERSION#)#

This also allows for more complex replacements in combination with #VERSION[x]#. For example, if you want the dotless major-minor version, you would do: #REPLACE([.],,#VERSION[:2]#)#.

Note how the value of x is [.] in some of the above examples. This is because x is treated as a regular expression and the dot is a special character in regular expressions so it needs to be put in [] to escape it. This allows for even more complex replacements like #REPLACE([.-_],,#VERSION#)# which removes all dots, dashes and underscores or something even more ridiculous as #REPLACE([0-9],,#VERSION#)# which replaces all numbers with nothing.

Anyway, flexibility is a good thing, especially when it comes to dealing with a million applications, each with its own conventions.

Generic version replacement

AppSnap supports text replacement to allow representing the version string in different parts of the database. This is a basic feature of AppSnap since most applications hard-code the version in the download URL or the file name among other places. Over time, AppSnap has added support for a bunch of commonly occurring version chunks.

I was recently adding some Firefox add-ons to the database when I ran into a version replacement not supported by AppSnap. I needed the 4th chunk of the version but AppSnap only went upto the third with #SUB_VERSION#. Since this design was not scaling, I’ve added a very generic method to access any chunk within the version string.

#VERSION[x]#

Leveraging, and hence, resembling Python’s excellent list slicing techniques, described in Dive into Python in sections 3.8 and 3.9, AppSnap can now pull out any chunk or chunks from the version string with one simple version replacement method. The examples below show how very flexible this new technique is:

Single chunk

• #MAJOR_VERSION# = #VERSION[0]#
• #MINOR_VERSION# = #VERSION[1]#
• #SUB_VERSION# = #VERSION[2]#

Multiple chunks

• #VERSION# = #VERSION[:]#
• #MAJORMINOR_VERSION# = #VERSION[:2]#
• #MAJORMINORSUB_VERSION# = #VERSION[:3]#

This new method supersedes all of the older, now deprecated methods. The older methods will be removed from a future version of AppSnap. The database has not been updated to use this new method as of yet since older versions of AppSnap which are still in use will be unable to handle the new system.

Next in line is a more generic replacement for the rest of the version replacements like #DOTLESS_VERSION# and #DASHTODOT_VERSION# that have the same problem as the ones above. I’m considering something like #REPLACE(x, y, string)# that will cover all those cases and a whole lot more. For example: #REPLACE(.,-,#VERSION#)# will do what #DOTTODASH_VERSION# does today. It will also allow something more complicated like #REPLACE(.,-,#VERSION[:2]#)# which is not available today.

All these changes basically allow AppSnap to be a lot more flexible when it comes to adapting to the variety of ways in which applications are released on the net.

Updates since 1.3.3-pre2

This post is a quick catch-up for a bunch of changes I’ve made in the last few days, ever since 1.3.3-pre2 came out. They are all available right away so use AppSnap’s update feature to get them.

Firefox and Thunderbird add-on support

For starters, AppSnap can now download and install Firefox and Thunderbird add-ons or extensions. This makes setting up these two applications a whole lot easier. Add-ons are very popular due to the functionality they add to the base product but they are tedious to setup one at a time. There is a huge ecosystem surrounding these platforms so supporting that is most sensible. I’ve already started adding the most recommended add-ons on the official Mozilla site to the AppSnap database.

Note, however, that only the installation step is supported. Neither app provides for an automated way to tinker with installed add-ons. AppSnap would have to parse extension directories, which are all over the place, delete extension folders and perform other tricks that are frankly excessive. Upgrades and uninstalls are handled well enough within Firefox and Thunderbird for AppSnap to have to manage that portion.

Lastly, all add-ons are installed as global extensions so all users inherit them. This is mainly because Thunderbird does not provide any other way to install extensions and I wanted to be consistent. This also means that both Firefox and Thunderbird can not be running while installing add-ons with AppSnap. If they are running, nothing gets installed and AppSnap will never know since the apps return 1 either way.

In the process of supporting add-ons, the basic foundation has been laid to support similar extensions for any other application. No example comes to mind at this time so if there is an application whose add-ons you would like to see supported, do let me know and I’ll look into it.

Verbose download status on the CLI

The CLI version of AppSnap can now display a cumulative download status with the -v flag. This was a much requested feature since many folks, including myself, use AppSnap on the command line quite a bit. Especially when downloading large application installers, it is good to provide some feedback to the user.

Detecting install directory

This feature was required in order for AppSnap to support add-ons. AppSnap can now detect the install directory of an application using the ‘instdir’ flag in the configuration. For now, this searches only the uninstall location in the registry so it is a little limited. Down the road, this will help detect application versions from specific files in the install directory. That should allow AppSnap to support version detection of several applications that do not store version information in the registry.

I am going to release 1.3.3 shortly since many folks are still stuck with 1.3.2, losing out on the long list of improvements made in the 1.3.3 pre-releases. Look out for that in another day or two.

Move to WordPress.com

Now that AppSnap supports the self-update feature, I’ve been adding functionality at a furious pace. The change log captures all the details but does not provide for a way to notify end users on a real-time basis. Also, the tone on a change log is quite boring. A blog is definitely a more exciting and fun way to communicate so here’s the official AppSnap blog. I’ve already moved all the older news items here so you can see what all happened in the past. There are just a few posts since April 2007, but from now on, you can expect posts every time anything changes with AppSnap. I’ve also created categories such as Code Changes, Database Changes, etc. in case you have specific areas of interest.

Prepending text to the News tiddler on the AppSnap website for each new post was cumbersome, requiring me to re-upload the site to SourceForge. It was also neither indexable or searchable by any search engine. WordPress just makes life easier. Many thanks, again, to the WordPress team for their fantastic product.