Help to fight Internet censorship!

Since Nov 1st 2012 the new Russian law implementing the Internet censorship comes into effect. This law allows several agencies of the Russian government to add any Internet site they consider “harmful to children” into the government-mandated block list. Russian Internet providers are legally obligated to block access to the sites which are present in this list. Quick summary of the law:

  • The sites could be blocked by either the court or one of the government agencies. The law currently allows blocking for three categories of web sites: child pornography, drug propaganda and the web sites about the suicide.
  • The law provides no oversight and no penalties for the government employees who add the specific site into the block list. Further, the list itself is secret and only available to the Internet providers.
  • The law requires the government to notify the site owner and let the owner to remove the content in three days. This, however, does not happen, and the sites get blocked without any advance warning.
  • It requires a court order to remove the site from the blocked list, while it could be added there simply by some government clerk.
  • The access is blocked for everyone, even the adults who don’t have any children. The block is mandatory.

Internet is the main vehicle fueling the democracy in Russia. Popular social networking sites  such as LiveJournal and Facebook/VKontakte are widely used by the opposition to coordinate the peaceful protests, uncover the major corruption scandals and simply exercise their free speech rights by sharing their opinions which are censored from the Russian government-maintained TV channels. Therefore a lot of Russians are worried the real purpose of a new law is to quickly shut down the resources the opposition uses to fight the Putin regime. During the first days the law went into effect it blocked a few political satire sites, and a site about the suicide prevention. This is just start.

It is a worrying trend when governments limit the Internet access for adults under the guise of “saving our children”. However we can help Russians to fight the Internet censorship.

What can you do:

  1. Spread the word! Tell others about the censorship and how to work around it.
  2. Set up Tor software or I2P software and run an exit node or an intermediate node. This will help the people to reach censored sites. We are running the Tor node here at Ulduzsoft.
  3. Donate to NoiseBridge or similar organizations which run the Tor exit nodes for everyone to use.
  4. Educate your friends  about the effects of the censorship on the society. Censorship is a very attractive option to any government, we must be vigilant to preserve our right to free speech!
Uncategorized Leave a comment

Reverse-engineering the KaraFun file format. Part 4, the encryption

So far all the files we have seen were not encrypted. However some users pointed out there are KFN files which are encrypted. While the encrypted files were still analysed and dumped properly, the output files were encrypted and hence unusable. But the users also confirmed that KaraFun player plays those files just fine, and does not ask any passwords from the user nor it requires Internet connection (and hence doesn’t download the keys from a server). Since our player need to support encrypted files too, this is something which we need to take care of now.

First let me start with a statement that reverse-engineering the file format encryption is typically a very difficult task even in a case like that, when the encryption key is not provided by user or the server, and therefore is embedded either into the software or into the file itself. Finding them out therefore requires reverse-engineering the actual software to find out where the keys are stored, and which algorithm is used. However as you see below due to a major flaw in the KaraFun software it is still possible to reverse-engineer even the encrypted files without even looking at the actual software!
Continue reading »

android, reverse engineering 13 Comments

Reverse-engineering the KaraFun file format. Part 3, the Song.ini file

This is quite simple. We look at the song.ini file and it is obvious immediately where the text and the timing information is as those are the only lines with enough numbers.
Continue reading »

android, reverse engineering Leave a comment

Reverse-engineering the KaraFun file format. Part 2, the directory

In the first part we found out the header format, and that it does not provide us with the directory location. However we know there must be a directory, as the KaraFun application must know where exactly in a file the files are stored, and how large are they. At minimum there should be the directory offset and either the total size or the number of files. At the first thought the DIFW header value may contain the number of files, and the MUSL value contains the directory offset (its value is 0x11D which is after 0x117). However if we check other KaraFun files at the same page, we would see that for some files the MUSL value is less than header length. Therefore it cannot be the offset, and probably is the music length in seconds. Nor DIFW is the number of files. A quick search for the JPEG signature “JFIF” finds out at least three JPG files, so there are more than two files in this archive.

So where it is the directory? Since the header length varies (because it uses the strings with variable length), it could be in one of two places. Either it is at the end of the file (not the case as we saw above), or it is supposed to follow the header directly. Let’s look carefully at the bytes following the header:
Continue reading »

android, reverse engineering Leave a comment

Reverse-engineering the KaraFun file format. Part 1, the header

Several of our users have expressed disappointment that our Ulduzsoft Karaoke Player for Android does not support the popular KaraFun Karaoke format. This format seem to be very popular in some countries, and unfortunately there seem to be no player on Android capable of playing those files. Even the KaraFun Android application does not play those files which is unfortunate. Therefore we decided to add support for this format.

The main issue we had to overcome was lack of any documentation on Internet about this popular format. There is no free open-source software supporting this format either. Therefore to support this file format I had to reverse-engineer it. Fortunately I have the relevant experience, and it was not a very difficult task. Then I decided to document those efforts for the readers to better understand how the reverse engineers work as there seem to be a lot of misunderstanding about the process. All I ultimately needed was a few KaraFun karaoke files. I didn’t even download any KaraFun software, and there was no need to use the editor. The whole format, including the encryption, was reverse-engineered by just looking at the file content.

Hopefully this article would be useful for the people who would like to support KaraFun files in their projects, or just curious about how the reverse engineering of file formats is done.

Continue reading »

android, reverse engineering Leave a comment

Ulduzsoft Karaoke Player for Android is out of Beta now!

Today we released Ulduzsoft Karaoke Player for Android version 1.12. It has been a beta for close to nine months, and it is labor time. The application seem to be well-received, with a competitive functionality and the most bugs seem to be fixed. We had a long discussion about the app monetization and decided to go with a freemium model with the actual app remaining free and ad-supported through Google AdMob, with the option to disable the ads by purchasing the license key which is also available on Android Market.

So it is time for the application to become self-sustained, when any further development depends on the user feedback and the income generated by the application purchases and the ads. Let’s see how well it goes.

android 1 Comment

Android dialog to choose a directory or file based on AlertDialog

This dialog could be used to let the user choose a file or directory. Since it is based on AlertDialog it doesn’t have to be instantiated through startActivity() and therefore could be used, for example, in a PrederenceDialog subclass. The provided code only selects the directories, but it is easy to modify it to select the specific files as well. This code is used in the Ulduzsoft Karaoke Player. Apache license.

Continue reading »

Uncategorized 3 Comments

Parsing ID3v2 tags in the MP3 files

This simple tag parser is very useful when you just need to get the basic information about the MP3 files, such as the title and the artist. Of course it could be extended to extract more information if necessary.

Apache license.
Continue reading »

android 3 Comments

Your personal GMail-like mail system: Sieve support

Dovecot has built-in support for the Sieve mail filtering language. It is very useful to do the server-side email processing such as:

  • Removing unwanted e-mail messages before they are delivered to your inbox;
  • Copying or moving e-mail messages to different folders;
  • Creating vacation autoresponses or any other kind of autoresponses
  • Configuring the actions above depending on message sender, recipient, subject, body, and so on.

This useful functionality is provided by the Dovecot mail delivery agent, dovecot-lda. However since we do not use it for delivery, it is not enabled. So the first step would be to enable it.

Do you have the Sieve plugin?

The Sieve language support plugin is not part of Dovecot source code. It is provided as a separate source which may or may not be packaged by your Linux distro. OpenSUSE packagers did the great job and dovecot bundled with OpenSUSE contains both Sieve and ManageSieve extensions. You may be less lucky in which case you may have to build it yourself.

An easy way to check if you have it is to see whether you have the configuration file /etc/dovecot/conf.d/90-sieve.conf in your dovecot installation. This file only comes with the Sieve plugin, so if you have this file, this means you have the plugin as well.

Enabling the Sieve plugin

The Sieve plugin could be enabled in the Dovecot configuration by editing the /etc/dovecot/conf.d/15-lda.conf file and adding sieve into the list of those mail plugins loaded by default:

protocol lda {
 # Space separated list of plugins to load (default is global mail_plugins).
 mail_plugins = $mail_plugins sieve
}

Enabling the dovecot-lda authentication agent

Unlike the getmail, the dovecot-lda does not know the Maildir path where to deliver the mail. Instead it receives the virtual user name as the command-line parameter, and queries the path from the dovecot authentication process. This makes the configuration simpler, but this also means the authentication should be enabled in the dovecot configuration. This is done by the auth-userdb service specified in the /etc/dovecot/conf.d/10-master.conf file and it is enabled by default. However in our case it needs a little tweaking since we run getmail under our mailman user and it will start the dovecot-lda process under the same user. By default this user does not have access to the dovecot authentication service and will not be able to query the necessary information. Therefore we need to change its permission by editing the auth-userdb section and changing the user the socket permissions as following:

unix_listener auth-userdb {
 mode = 0600
 user = mailman
 #group =
}

Then restart dovecot. The dovecot-lda can now be used.

Reconfiguring getmail to use dovecot-lda for mail delivery

To use the dovecot-lda for mail delivery getmail needs to be reconfigured as it needs to be explicitly told to use the mail delivery agent instead of doing direct delivery. This is done by modifying the [destination] section in the getmailrc configuration file. The new section should look like the one below:

[destination]
type = MDA_external
path = /usr/lib/dovecot/dovecot-lda
arguments = ("-f", "%(sender)", "-d", "pers" )

As you see the path to the destination mailbox is now specified anymore. Instead the -d command-line option followed up by the virtual user name (which in our case is pers) is used to tell the dovecot-lda which user this email is for. The mail directory is looked up through the dovecot authentication agent.

Now send yourself the test e-mail and verify that it gets properly delivered. Check the /var/log/mail in case of any errors, which will likely be triggered by either the wrong virtual username or lack of permissions to access the dovecot authentication process socket.

Managing the Sieve scripts from your e-mail client

Bundled together with Sieve, Dovecot provides a nice way to manage the Sieve scripts right from your e-mail client assuming it has necessary capabilities. KMail can do it natively, Thunderbird can do it with the Sieve add-on.

This functionality must be enabled in Dovecot to be supported, but it is fairly easy to enable. Just edit the /etc/dovecot/conf.d/20-managesieve.conf file and uncomment the line started with protocols:

# Uncomment to enable managesieve protocol:
protocols = $protocols sieve

Then restart dovecot. That’s all.

Sample Sieve script

An example of Sieve script which does various tasks:

require ["fileinto", ["vacation"];
if header :is "From" "john.doe@high.profile.business.com"
{
    # Store business emails from john.doe in a Work/JohnCompany folder
    fileinto "Work.JohnCompany";
}
elsif header :is "From" "buyviagra@spam.com"
{
    # Just delete that spam
    discard;
    stop;
}
elsif header :contains "From" "john.doe@"
{
    # If this is email from john.doe sent from any other of his accounts, move it to Parties folder
    fileinto "Parties";
}
else
{
    # Tell everyone else I'm on vacation
    vacation

    # Reply at most once a day to a same sender
    :days 1

    # The auto-reply subject
    :subject "Out of country: vacation"

    # The reply text
    "I will be out of country for a week.
     Please call my mobile +X XXX XXXXXXXXXX for urgent matters.
Best regards
John Smith";
}

									
Email , , Leave a comment

Your personal GMail-like mail system: the web interface

So you got the system which is as good as GMail, but you also want to have the Web interface. This comes handy when you’re in the middle of changing e-mail clients, or you’re traveling without your laptop and want to check your e-mail from the public Internet cafe. So you want to have the Webmail access to your mail – which, obviously, should not disrupt your regular e-mail flow.

After trying several free solutions I decided to use the RoundCube. It was simple enough to install and configure, and it has the best Web interface I’ve seen so far around free webmail solutions.

This post assumes you have installed and configured the Apache with mod_php5 on your Linux web server, or you know how to do so.

Installing RoundCube

The installation itself is very simple and consist of pretty straightforward steps:

  • Download the latest stable RoundCube version from the project SourceForge page
  • Create a roundcube directory in your web server root and unpack the archive content there. It creates the roundcubemail-0.x directory.
  • Move all the files from there. Make sure you moved .htaccess file as well. Try rmdir roundcubemail-0.* – if you get “Directory not empty” error you forgot it. You should now have the whole installation in your /roundcube directory.
  • Password-protect the directory if you’re security-aware, just in case there are security issues with RoundCube which you won’t find in time about. This could be done by adding the following content to the .htaccess file:
AuthType Basic
AuthName "RoundCube install"
AuthUserFile user.passwd
AuthBasicProvider file
AuthUserFile /path/to/auth/file/outside/the/web/directory
Require valid-user

and create the auth file specified in the AuthUserFile by running htpasswd2. I also suggest making it readable by the web server only:

sudo htpasswd2 -c /path/to/auth/file/outside/the/web/directory roundcube
  • The line below adds the roundcube user with the password you specified. Remember it.
  • Make the roundcube temp and logs directories writable by the web server
<strong><span style="font-size: medium;">Creating the SQLite database</span></strong>

I suggest to use the SQLite database for roundcube. You’re not going to use it often and heavily so there is no need to run the heavy database daemon.

Type the following:

> mkdir sqlite
> sqlite -init SQL/sqlite.initial.sql sqlite/sqlite.db
> sudo chown -R wwwrun sqlite

You need this because to use SQLite the process requires write permission on the directory the database is stored (to create journal files).

The version of RoundCube I installed had an error in the SQL script. If you get the similar error message:

CREATE UNIQUE INDEX ix_searches_user_type_name (user_id, type, name);
SQL error: near "(": syntax error
SQLite version 2.8.17
Enter ".help" for instructions
sqlite>

Just press “Ctrl+D” to exit sqlite prompt, and edit the SQL/sqlite.initial.sql file. Find the following line:

CREATE UNIQUE INDEX ix_searches_user_type_name (user_id, type, name);

and edit it so it looks like that:

CREATE UNIQUE INDEX ix_searches_user_type_name <span style="background-color: #ffd700;">ON searches</span> (user_id, type, name);

then delete the sqlite/sqlite.db file and try again.

Configuring RoundCube

  • Open the browser and point it to http://<your site>/roundcube/installer/
  • Login as the user roundcube with the password you specified above when running the htpasswd2 line.
  • Press the “Start installation” button below.
  • The next screen verifies that you have everything which is needed to run RoundCube. If it lists missing modules, install them. For the “available databases” make sure it says “OK” near SQLite; ignore any error with other databases there. Press Next.
  • In the next screen scroll to the Database setup. Select SQLite in the listbox. Type the full path (including the file name) of the sqlite.db you just created in the previous step there. Leave other fields intact.
  • In the IMAP settings add your hostname/IP as ssl://your.address.com
  • Do the same in SMTP settings assuming you did set up a local SMTP server. For SMTP you have to specify the port, use 587. Leave the smtp_user/smtp_pass empty and check the “Use the current IMAP username and password for SMTP authentication” checkbox below.
  • The next screen will show you two configuration files and asks you to copy them to the server. Download them in your browser and scp them to the web server into the roundcube/config/ directory.
  • Scroll down the screen and find the Continue button somewhere in the middle (not on the bottom). Press it.
  • On this screen it verifies the configuration and database. Should be OK everywhere. You may also test your IMAP/SMTP there. Don’t worry if IMAP port is shown as 143, if you set it as ssl:// it will connect properly.
  • Once everything is fine, go back to your console and delete the installer folder from your roundcube installation.

Using RoundCube

Open the http://your.site.com/roundcube and login with your IMAP credentials. You should be able to login successfully. If RoundCube still says “login failed” and you’re sure your credentials are correct, most likely you forgot to change permissions on the sqlite directory in the database file. Both should be owned by the user your web server is run under.

Once you log in you should see the message list. Messages are opened using the double click. Enjoy!

Email Leave a comment