Adding applications to AppSnap

Adding applications to AppSnap takes longer then you would think. Each application needs to be downloaded, installed and then uninstalled in order to get all the information that AppSnap needs. On an average, it takes a good 10-15 minutes per application, excluding any download time. That’s not too bad when you start out but after a point, it is too tedious to deal with.

With 1.3.3, AppSnap crossed the 200 mark in terms of supported applications. It took almost a year and a half to get that far and that reflects the pains associated with growing the database. Considering I am the only guy adding applications as well as enhancing AppSnap, both on a part-time basis, it makes it that much slower. Something had to change.

It is not possible to automate database additions since each application is unique. However, it is possible to speed up the steps that it takes add an application. To make life a little easier, I added the -a flag to the AppSnap CLI to facilitate adding applications. Using -a, it now takes at most five minutes to get an application added to AppSnap (excluding download time). If you haven’t noticed already, the database now has 285 applications and counting. That is more than 80 applications in 12 days. Not bad.

Adding applications is not only faster, but a lot more fun. Indeed, it is the small things that make a big difference.

Generic version replacement – II

I have successfully implemented a replacement for #DOTLESS_VERSION#, #DASHTODOT_VERSION#, etc. as mentioned previously. This successfully deprecates the following existing version replacements as follows:-

• #DOTLESS_VERSION# = #REPLACE([.],,#VERSION#)#
• #DASHTODOT_VERSION# = #REPLACE(-,.,#VERSION#)#
• #DOTTOUNDERSCORE_VERSION# = #REPLACE([.],_,#VERSION#)#
• #DOTTODASH_VERSION# = #REPLACE([.],-,#VERSION#)#

This also allows for more complex replacements in combination with #VERSION[x]#. For example, if you want the dotless major-minor version, you would do: #REPLACE([.],,#VERSION[:2]#)#.

Note how the value of x is [.] in some of the above examples. This is because x is treated as a regular expression and the dot is a special character in regular expressions so it needs to be put in [] to escape it. This allows for even more complex replacements like #REPLACE([.-_],,#VERSION#)# which removes all dots, dashes and underscores or something even more ridiculous as #REPLACE([0-9],,#VERSION#)# which replaces all numbers with nothing.

Anyway, flexibility is a good thing, especially when it comes to dealing with a million applications, each with its own conventions.

Generic version replacement

AppSnap supports text replacement to allow representing the version string in different parts of the database. This is a basic feature of AppSnap since most applications hard-code the version in the download URL or the file name among other places. Over time, AppSnap has added support for a bunch of commonly occurring version chunks.

I was recently adding some Firefox add-ons to the database when I ran into a version replacement not supported by AppSnap. I needed the 4th chunk of the version but AppSnap only went upto the third with #SUB_VERSION#. Since this design was not scaling, I’ve added a very generic method to access any chunk within the version string.

#VERSION[x]#

Leveraging, and hence, resembling Python’s excellent list slicing techniques, described in Dive into Python in sections 3.8 and 3.9, AppSnap can now pull out any chunk or chunks from the version string with one simple version replacement method. The examples below show how very flexible this new technique is:

Single chunk

• #MAJOR_VERSION# = #VERSION[0]#
• #MINOR_VERSION# = #VERSION[1]#
• #SUB_VERSION# = #VERSION[2]#

Multiple chunks

• #VERSION# = #VERSION[:]#
• #MAJORMINOR_VERSION# = #VERSION[:2]#
• #MAJORMINORSUB_VERSION# = #VERSION[:3]#

This new method supersedes all of the older, now deprecated methods. The older methods will be removed from a future version of AppSnap. The database has not been updated to use this new method as of yet since older versions of AppSnap which are still in use will be unable to handle the new system.

Next in line is a more generic replacement for the rest of the version replacements like #DOTLESS_VERSION# and #DASHTODOT_VERSION# that have the same problem as the ones above. I’m considering something like #REPLACE(x, y, string)# that will cover all those cases and a whole lot more. For example: #REPLACE(.,-,#VERSION#)# will do what #DOTTODASH_VERSION# does today. It will also allow something more complicated like #REPLACE(.,-,#VERSION[:2]#)# which is not available today.

All these changes basically allow AppSnap to be a lot more flexible when it comes to adapting to the variety of ways in which applications are released on the net.

Updates since 1.3.3-pre2

This post is a quick catch-up for a bunch of changes I’ve made in the last few days, ever since 1.3.3-pre2 came out. They are all available right away so use AppSnap’s update feature to get them.

Firefox and Thunderbird add-on support

For starters, AppSnap can now download and install Firefox and Thunderbird add-ons or extensions. This makes setting up these two applications a whole lot easier. Add-ons are very popular due to the functionality they add to the base product but they are tedious to setup one at a time. There is a huge ecosystem surrounding these platforms so supporting that is most sensible. I’ve already started adding the most recommended add-ons on the official Mozilla site to the AppSnap database.

Note, however, that only the installation step is supported. Neither app provides for an automated way to tinker with installed add-ons. AppSnap would have to parse extension directories, which are all over the place, delete extension folders and perform other tricks that are frankly excessive. Upgrades and uninstalls are handled well enough within Firefox and Thunderbird for AppSnap to have to manage that portion.

Lastly, all add-ons are installed as global extensions so all users inherit them. This is mainly because Thunderbird does not provide any other way to install extensions and I wanted to be consistent. This also means that both Firefox and Thunderbird can not be running while installing add-ons with AppSnap. If they are running, nothing gets installed and AppSnap will never know since the apps return 1 either way.

In the process of supporting add-ons, the basic foundation has been laid to support similar extensions for any other application. No example comes to mind at this time so if there is an application whose add-ons you would like to see supported, do let me know and I’ll look into it.

Verbose download status on the CLI

The CLI version of AppSnap can now display a cumulative download status with the -v flag. This was a much requested feature since many folks, including myself, use AppSnap on the command line quite a bit. Especially when downloading large application installers, it is good to provide some feedback to the user.

Detecting install directory

This feature was required in order for AppSnap to support add-ons. AppSnap can now detect the install directory of an application using the ‘instdir’ flag in the configuration. For now, this searches only the uninstall location in the registry so it is a little limited. Down the road, this will help detect application versions from specific files in the install directory. That should allow AppSnap to support version detection of several applications that do not store version information in the registry.

I am going to release 1.3.3 shortly since many folks are still stuck with 1.3.2, losing out on the long list of improvements made in the 1.3.3 pre-releases. Look out for that in another day or two.

AppSnap 1.3.1 on it’s way

On receiving bug reports of the AppSnap CLI crashing, I first thought it was specific to Windows Vista. Turns out it is broken in XP as well. I’m not seeing the issue if I run AppSnap from source in a Python 2.5.1 environment. However, on creating the executable using Py2Exe, I see the problem consistently. If I create an executable using Python 2.4.4 though, the issue is no longer seen.

I also tried PyInstaller to see if it was a Py2Exe bug but it failed too. For some reason, AppSnap was crashing Python 2.5.1 in the threading module when packaged. After several hours of debugging, I realized that I was not join()’ing all threads and that upset Python. Didn’t find anything on Google about this but luckily, it was a simple fix.

Keep a look out for a bug fix release shortly.