Reverse-engineering the KaraFun file format. Part 1, the header

Several of our users have expressed disappointment that our Ulduzsoft Karaoke Player for Android does not support the popular KaraFun Karaoke format. This format seem to be very popular in some countries, and unfortunately there seem to be no player on Android capable of playing those files. Even the KaraFun Android application does not play those files which is unfortunate. Therefore we decided to add support for this format.

The main issue we had to overcome was lack of any documentation on Internet about this popular format. There is no free open-source software supporting this format either. Therefore to support this file format I had to reverse-engineer it. Fortunately I have the relevant experience, and it was not a very difficult task. Then I decided to document those efforts for the readers to better understand how the reverse engineers work as there seem to be a lot of misunderstanding about the process. All I ultimately needed was a few KaraFun karaoke files. I didn’t even download any KaraFun software, and there was no need to use the editor. The whole format, including the encryption, was reverse-engineered by just looking at the file content.

Hopefully this article would be useful for the people who would like to support KaraFun files in their projects, or just curious about how the reverse engineering of file formats is done.

A side note: since I use Linux, the whole reverse-engineering process happens by using the command-line tools. I also wrote some Java code to speed the things up. Therefore if you’re using Windows, you might have to install Cygwin and find out a few tools, notably hexdump.

Even before we see our first KaraFun file, we already have some information about it. We know this file contains Karaoke, and this means it contains at least the music and lyrics. Therefore it is a container file type, like an archive.

Now, in every container file there are at least three pieces of information:

  • The container file header which identifies the things such as the container type, version, creation/update date and so on. Typically it is located at the beginning of the file.
  • The container directory, which lists the files stored in the container. At minimum it contains two fields, the offset in the container file and the length. It can also contain more fields, such as checksum, flags (whether the file is packed/encrypted and which way), file create/modification timestamps, name and so on. The directory can be either centralized (i.e. the information of all the files is stored together) or decentralized (each file is prefixed by its own directory)
  • The stored files.

Knowing what to expect, we get the first KFN file. I used the song Chua biet ro.kfn available from the http://firstvietnamesechurch.com/nhc/download-karafuns and stored it as 1.kfn

The first thing to do is to hex-dump the file:

> hexdump -C 1.kfn | less
 00000000 4b 46 4e 42 44 49 46 4d 01 02 00 00 00 44 49 46 |KFNBDIFM.....DIF|
 00000010 57 01 02 00 00 00 47 4e 52 45 01 74 00 00 00 53 |W.....GNRE.t...S|
 00000020 46 54 56 01 53 00 0a 01 4d 55 53 4c 01 1d 01 00 |FTV.S...MUSL....|
 00000030 00 41 4e 4d 45 01 0c 00 00 00 54 59 50 45 01 00 |.ANME.....TYPE..|
 00000040 00 00 00 46 4c 49 44 02 10 00 00 00 00 00 00 00 |...FLID.........|

Here we can see that the KFN container starts with the header with several sections. Let’s look a little further:

 00000200 00 00 00 00 00 00 08 00 00 00 6e 62 72 34 2e 6a |..........nbr4.j|
 00000210 70 67 03 00 00 00 26 a1 01 00 aa 9a 8f 00 26 a1 |pg....&.......&.|
 00000220 01 00 00 00 00 00 08 00 00 00 6e 62 72 35 2e 6a |..........nbr5.j|
 00000230 70 67 03 00 00 00 88 92 00 00 d0 3b 91 00 88 92 |pg.........;....|
 00000240 00 00 00 00 00 00 08 00 00 00 53 6f 6e 67 2e 69 |..........Song.i|
 00000250 6e 69 01 00 00 00 f3 12 00 00 58 ce 91 00 f3 12 |ni........X.....|
 00000260 00 00 00 00 00 00 ff fa 92 60 9e d6 00 00 02 45 |.........`.....E|

This looks like the cenrtal directory, it even got the file names. Looking at it, seems like Song.ini is the last file in the directory. Since it is an INI file it is likely to be the text file, so let’s take a look at the KFN file end:

 0091e340 e1 bb 8d 69 20 c4 91 c3 a0 6e 67 2e 2e 0d 0a 49 |...i ....ng....I|
 0091e350 6e 53 79 6e 63 3d 31 0d 0a 0d 0a 5b 4d 50 33 4d |nSync=1....[MP3M|
 0091e360 75 73 69 63 5d 0d 0a 4e 75 6d 54 72 61 63 6b 73 |usic]..NumTracks|
 0091e370 3d 31 0d 0a 54 72 61 63 6b 30 3d 2c 30 2c 2d 31 |=1..Track0=,0,-1|
 0091e380 2c 30 39 2d 20 43 68 75 61 20 62 69 65 74 20 72 |,09- Chua biet r|
 0091e390 6f 2c 30 39 2d 43 68 75 61 20 62 69 65 74 20 72 |o,09-Chua biet r|
 0091e3a0 6f 2d 68 61 74 2d 6d 69 78 2e 6d 70 33 0d 0a 0d |o-hat-mix.mp3...|
 0091e3b0 00 |.|

We see some text. Therefore we can conclude that the files are stored as-is, they are not packed nor encrypted, and there is no footer at the end of the container file. Our task would be easy.

Now let’s go back to the header and look at it more carefully:

 00000000 4b 46 4e 42 44 49 46 4d 01 02 00 00 00 44 49 46 |KFNBDIFM.....DIF|
 00000010 57 01 02 00 00 00 47 4e 52 45 01 74 00 00 00 53 |W.....GNRE.t...S|
 00000020 46 54 56 01 53 00 0a 01 4d 55 53 4c 01 1d 01 00 |FTV.S...MUSL....|
 00000030 00 41 4e 4d 45 01 0c 00 00 00 54 59 50 45 01 00 |.ANME.....TYPE..|
 00000040 00 00 00 46 4c 49 44 02 10 00 00 00 00 00 00 00 |...FLID.........|
 00000050 00 00 00 00 00 00 00 00 00 00 00 00 54 49 54 4c |............TITL|
 00000060 02 0c 00 00 00 43 68 75 61 20 62 69 65 74 20 72 |.....Chua biet r|
 00000070 6f 41 52 54 53 02 0e 00 00 00 41 75 67 75 73 74 |oARTS.....August|
 00000080 69 6e 6f 20 48 6f 61 69 41 4c 42 4d 02 0c 00 00 |ino HoaiALBM....|
 00000090 00 43 68 75 61 20 62 69 65 74 20 72 6f 43 4f 4d |.Chua biet roCOM|
 000000a0 50 02 0e 00 00 00 41 75 67 75 73 74 69 6e 6f 20 |P.....Augustino |
 000000b0 48 6f 61 69 53 4f 52 43 02 1c 00 00 00 31 2c 49 |HoaiSORC.....1,I|
 000000c0 2c 30 39 2d 20 43 68 75 61 20 62 69 65 74 20 72 |,09- Chua biet r|
 000000d0 6f 2d 6d 69 78 2e 6d 70 33 54 52 41 4b 02 01 00 |o-mix.mp3TRAK...|
 000000e0 00 00 31 52 47 48 54 01 00 00 00 00 50 52 4f 56 |..1RGHT.....PROV|
 000000f0 01 00 00 00 00 49 44 55 53 02 10 00 00 00 20 20 |.....IDUS..... |
 00000100 20 20 20 20 20 20 20 20 20 20 20 20 20 20 45 4e | EN|
 00000110 44 48 01 ff ff ff ff 09 00 00 00 1b 00 00 00 30 |DH.............0|

Since the header contains the strings, it cannot use the fixed length fields (like having each field to be exactly 10 bytes long) as it would waste the disk and memory space, so there must be some other way to find out how large a specific header field is. Take a closer look at the lines 50-70:

 00000050 00 00 00 00 00 00 00 00 00 00 00 00 54 49 54 4c |............TITL|
 00000060 02 0c 00 00 00 43 68 75 61 20 62 69 65 74 20 72 |.....Chua biet r|
 00000070 6f 41 52 54 53 02 0e 00 00 00 41 75 67 75 73 74 |oARTS.....August|

The song name from the web site is stated as “Chua biet ro”. Since it is prefixed with the TITL string, we can reasonably assume the TITL means “Title”. However the song name could have any length, how does the parser know there to terminate the string? There are several ways to store the strings in the memory. One is a C/C++ way (the way the strings are stored in C and C++ programming languages) when the string is terminated by the 0x00 value. Another one is the Pascal way (if you didn’t know, Pascal is another programming language) when a string stored was prefixed with the string length stored in the first 2 or  4 bytes. We do not see 0x00 at the end of the title, and two hex characters in front of the title are 0x00 0x00 which can’t encode the string length.  However if we look further at four underscored bytes which are followed by the string, they are 0x0C 0x00 0x00 0x00. Using the little-endian encoding it means 0x0000000C, or 12. Which is exactly the length of “Chua biet ro”. Great!

The only question is what the 0x02 between the TITL and the length means. To find out we look at other fields, and list the prefixes, the intermediate value and the field length which we can easily predict since the field ends when a new header starts:

 DIFW - value 0x01 - followed by 4 bytes, value 0x00000002
 GNRE - value 0x01 - followed by 4 bytes, value 0x00000074
 SFTV - value 0x01 - followed by 4 bytes, value 0x010A0053
 MUSL - value 0x01 - followed by 4 bytes, value 0x0000011D
 ANME - value 0x01 - followed by 4 bytes, value 0x0000000C
 TYPE - value 0x01 - followed by 4 bytes, value 0x000000000
 FLID - value 0x02 - followed by 4 bytes length (0x10) followed by 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 TITL - value 0x02 - followed by 4 bytes length (0x0C) followed by "Chua biet ro"

Now we can describe the header structure of the KFN file. The first four bytes of the KFN file starts with the “KFNB” characters, followed by the headers. Each header has the following structure: four-character field name (same as used by RIFF container) followed by the flag. If the flag value is 0x01, then the next 4 bytes after the flag contain the value for this field. If the flag value is 0x02, then the next 4 bytes contain the length of the string which follows up the length bytes. The header ends with ENDH field.

Let’s write a small Java program to dump the header in a human-readable form:

import java.io.File;
import java.io.IOException;
import java.io.FileNotFoundException;
import java.io.RandomAccessFile;
import java.nio.ByteBuffer;
import java.nio.charset.Charset;

class KFNDumper
{
	public boolean parse( String fontFilename )
	{
		try
		{
			m_file = new RandomAccessFile( fontFilename, "r" );

			// Read the file signature
			String signature = new String( readBytes(4) );

			if ( !signature.equals("KFNB") )
				return false;

			// Read the header
			while ( true )
			{
				signature = new String( readBytes(4) );
				int type = readByte();
				int len_or_value = readDword();

				switch ( type )
				{
					case 1:
						System.out.println( signature + ", type 1, value " + String.format("%x", len_or_value));
						break;

					case 2:
						byte[] buf = readBytes( len_or_value );
						System.out.println( signature + ", type 2, length " + len_or_value + ", hex: " + dumpHex( buf ) + ", string: \"" + Charset.forName( "UTF-8" ).decode( ByteBuffer.wrap( buf ) ).toString() + "\"" );
						break;
				}

				if ( signature.equals("ENDH") )
					break;
			}

			System.out.println( "Header ends at offset " + m_file.getFilePointer() );
			return true;
		}
		catch (IOException e)
		{
			// Most likely a corrupted font file
			return false;
		}
	}

	// KFN file; must be seekable
	private RandomAccessFile m_file = null;

	// Helper I/O functions
	private int readByte() throws IOException
	{
		return m_file.read() & 0xFF;
	}

	private int readWord() throws IOException
	{
		int b1 = readByte();
		int b2 = readByte();

		return b2 << 8 | b1;
	}

	private int readDword() throws IOException
	{
		int b1 = readByte();
		int b2 = readByte();
		int b3 = readByte();
		int b4 = readByte();

		return b4 << 24 | b3 << 16 | b2 << 8 | b1;
	}

	private byte [] readBytes( int length ) throws IOException
	{
		byte [] array = new byte [ length ];

		if ( m_file.read( array ) != length )
			throw new IOException();

		return array;
	}

	private String dumpHex( byte [] array )
	{
		String out = "";

		for ( int i = 0; i < array.length; i++ )
		{
			if ( i > 0 )
				out += " ";

			out += String.format("%02X", array[i] & 0xFF);
		}

		return out;
	}

	private String readUtf8String( int length ) throws IOException
	{
		// Allocate the buffer and read into it
		byte[] buf = readBytes( length );

		// And decode the UTF-8 string
		return Charset.forName( "UTF-8" ).decode( ByteBuffer.wrap( buf ) ).toString();
	}

	private String readUtf8String() throws IOException
	{
		// First four bytes define the length
		return readUtf8String( readDword() );
	}

    public static void main( String [] args ) throws Exception
    {
		if ( args.length == 0 )
		{
			System.out.println( "Usage: app <KFN file>\n" );
			return;
		}

		KFNDumper dumper = new KFNDumper();
		dumper.parse( args[0] );
    }
}

Save it into the file KFNDumper.java, compile it and run:

> javac KFNDumper.java
> java KFNDumper 1.kfn
 DIFM, type 1, value 2
 DIFW, type 1, value 2
 GNRE, type 1, value 74
 SFTV, type 1, value 10a0053
 MUSL, type 1, value 11d
 ANME, type 1, value c
 TYPE, type 1, value 0
 FLID, type 2, length 16, hex: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00, string: ""
 TITL, type 2, length 12, hex: 43 68 75 61 20 62 69 65 74 20 72 6F, string: "Chua biet ro"
 ARTS, type 2, length 14, hex: 41 75 67 75 73 74 69 6E 6F 20 48 6F 61 69, string: "Augustino Hoai"
 ALBM, type 2, length 12, hex: 43 68 75 61 20 62 69 65 74 20 72 6F, string: "Chua biet ro"
 COMP, type 2, length 14, hex: 41 75 67 75 73 74 69 6E 6F 20 48 6F 61 69, string: "Augustino Hoai"
 SORC, type 2, length 28, hex: 31 2C 49 2C 30 39 2D 20 43 68 75 61 20 62 69 65 74 20 72 6F 2D 6D 69 78 2E 6D 70 33, string: "1,I,09- Chua biet ro-mix.mp3"
 TRAK, type 2, length 1, hex: 31, string: "1"
 RGHT, type 1, value 0
 PROV, type 1, value 0
 IDUS, type 2, length 16, hex: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20, string: " "
 ENDH, type 1, value ffffffff
 Header ends at offset 279

Most fields are pretty descriptive such as TITL (song title), ARTS (song artist), TRAK (the track number) and seem to originate from the MP3 ID3 tag format. Nevertheless we miss a key piece of information – the directory location.We need to find it out.

Continue to Part 2

This entry was posted in android, reverse engineering.

11 Responses to Reverse-engineering the KaraFun file format. Part 1, the header

  1. Andrey Saulin says:

    You are genius!!!
    Thank you very much…

  2. lalitha says:

    Hello,
    U all seem to have done a lot of work on the karafun. I have an issue with including Indian Languages in the lyrics. Karafun doesn’t support the font. Can you please guide me on this. (I am totally illiterate in the context of programming and computer languages). Hoping fervently that I would get a solution to my problem here.

    Thank you

  3. Joseph Kennedy says:

    George,

    Your article on reverse engineering the kfn audio file format was an enjoyable read. I have been using Karafun player for Windows for the past few months and it seems to download audio files to a cache folder at /ProgramData/Recisio/KaraFun2/Cache and the format does not match the kfn format you figured out, so I think perhaps these cached music files are either encrypted and unlocked by a key that is tied to being logged into the software (that would make sense since the software rejects its whole cache as invalid after certain network related events). I’m wondering where one finds the kfn files you reverse engineered, for I would love to download these songs I’m renting and play them with other software if there was a way other than using these cache files that only Karafun’s player can read.

  4. Jean Garon says:

    J’ai installé la version 1.31 de Ulduzsoft sur mon Samsung. Il joue les formats Kar, Midi et CDG sans problèmes, mais pas les fichiers KFN comme spécifié sur votre site. J’ai aussi acheté la version payante 1.5, mais ça ne fonctionne pas plus.
    Où est le problème? Y-a-t-il quelque chose que je ne fais pas comme il faut?
    Merci de me répondre.

  5. Víctor Juárez Cabrera says:

    Thank you very much for your code, it helped me a lot to be able to generate kfn files from a java application. Now with the software to remove the voice from an audio it allows you to automate many processes and create karaokes, if you like one day I will show you my application.

Leave a Reply to George Cancel reply

Your email address will not be published. Required fields are marked *


Warning: Use of undefined constant XML - assumed 'XML' (this will throw an Error in a future version of PHP) in /home/ulduzs/public_html/wp-content/plugins/wp-syntaxhighlighter/wp-syntaxhighlighter.php on line 1048