I would like to acknowledge Jed Wing for reverse-engineering this file and Razvan Cojocaru for pointing out clarity issues in this section.

This file stores information for the full-text search function of HTML Help. It prevents the need for viewers of CHM files to search each and every topic in the CHM to find words or phrases. This speeds up searching considerably, since the index that this file stores, contains data on which words occur in which files and at which locations.

This file will be empty when Full-text search has been turned off or when no files have been indexed. The CHM compiler will only index those files that contain .h in their names. All word sorting, processing and storage is done case-insensitively and is not case-preserving. If you have a word longer than 99 characters in a HTML file then it seems the indexing routines will die during indexing of that file and then skip on to the next one.

The full-text search data is structured as a tree that is more similar to the ITSP directory than the BTree file. The file begins with a header. This is followed by index nodes, leaf nodes and word location codes (WLCs). The index and leaf nodes are a fixed size (specified in the header) and the WLCs are variably sized (specified in the leaf nodes). There is a WLC block just before each leaf node and the pairs of WLC blocks and leaf nodes are interspersed with index nodes. Index nodes only point to nodes (index or leaf) that are further down the tree and at the same time located colser to the start of the file. The index nodes are directly after the last of their child nodes in the file. The structure of this format is best explained by the following diagram of a ficticious $FIftiMain file. For a more extensive set of examples see the example section

Figure 5.1. A pictorial description of the $FIftiMain format from CHM files.

This file makes use of two different ways of encoding integers in variable length fields; the so called scale and root method and a variant of the ENCINT method used in the PMGL/PMGI directory chunks from the ITSF format. For the ENCINTs in this file the bytes are stored least significant first (little endian), whereas in the PMGL/PMGI chunks they are stored most significant first (big endian).

For CHM viewers that want to quickly find out which files a word is in, the procedure is as follows; read the header, seek to the root index node, search the root index node for a word that is less than or equal to the desired, descend to the next index level, repeat the previous two steps as many times as the tree is deep, then search the resulting leaf node until the desired word is found, read the correct part of the WLCs for that leaf node and extract the topic numbers for that word. Once all the topic numbers have been found for the desired words, make sure there are no duplicates and extract the titles and urls from the #TOPICS, #URLTBL, #URLSTR and #STRINGS files.

For sample code for decoding $FIftiMain files, see chmdeco, xCHM or dump_fiftimain.c from the old html versions of this specification.

The file begins with a header that is 0x400 bytes in length:

The index nodes begin with a WORD indicating the length of free space at the end of the node. This is followed by the entries, which fill up as much of the index node as possible.

The leaf nodes begin with a short header, which is followed by entries:

Each WLC block is made up of bits packed as tightly as possible and is right-padded with 0s to a full byte. The fields are encoded as the scale and root variable-length integer format described below, with the parameters taken from the initial header. "Delta coding" is also used in a couple of places to reduce the size of the codes -- that is, the first value is stored verbatim, and subsequent values are stored as a delta or difference from the previous value.

The scale and root method of integer encoding needs two parameters, which I'll call s (scale) and r (root size). In the context of $FIftiMain files, s always appears to be '2', but any other power of 2 could also work (and might be used in some rare cases).

The integer is encoded as two parts, p (prefix) and q (actual bits). p determines how many bits are stored, as well as implicitly determining the high-order bit of the integer. To encode an integer, p starts out as a single 0. If the integer fits in r bits, you're done. If the integer fits in r+1 bits (i.e. r-th bit is set, counting from 0), prepend a 1 to the p and store the low r bits of the integer in q. Otherwise, while the integer does not fit in the allotted space, prepend a bit to p, and increase the size of q by one bit. It's hard to see from the description, but an example will make it more clear. Using s=2, r=3:

A scale other than 2 has never been seen, so it is hard to say how s/r encoding works when s=4, etc. The following is how it might work using s=4, r=2 i.e. a base-4 digit is added each time, meaning two bits added each time.:

Of course, this is all wild speculation, since examples with s other than 2 haven't been seen... But the codes do work this way (i.e. prepending a 1 to the prefix multiplies the additive value 'b' by s and adds another log2(s) bits.)

This is all fairly complex, so some examples will be extremely useful here. First a short ficticious example, then an example from the real world.

The following table is a ficticious example of how the words appear in leaf nodes. The storage of words in index nodes is the same as in leaf nodes, except some fields are gone.

This example is taken from a copy of windows.chm, the system documentation apparently distributed with some version of Windows 98:

Hex dump of two leaf node entries:

000223d:                                        02 00 31  ...0...........1
0002240: 00 0a 03 04 00 00 00 00 1d 01 01 01 01 20 04 00  ............. ..
0002250: 00 00 00 03

The fields of these two entries are as follows:

Scanning over to offset 0x403 in the file, we see:

0000403:          f9 f4 60 86 b8 ea 6a 00 ed 78 00 2d c0
0000410: f8 d7 28 2c f0 f6 dc c8 ce 66 61 80 87 02 00 00
0000420: f9 f4 40

Broken out, these WLC entries are:

1          <10, 1027, 29>:  f9 f4 60 86 b8 ea 6a 00 ed 78 00 2d c0 f8 d7 28
                            2c f0 f6 dc c8 ce 66 61 80 87 02 00 00
1 (TITLE)  <1, 1056, 3>:    f9 f4 40

Now, the parameters for the WLC in this file are 2/2, 2/1, 2/5. Here is a quick reference table for the codes:

p       value       q (bits)
2/1:
0:      0-1         1
10:     2-3         1
110:    4-7         2
1110:   8-15        3
11110:  16-31       4
111110: 32-63       5
2/2:
0:      0-3         2
10:     4-7         2
110:    8-15        3
1110:   16-31       4
11110:  32-63       5
111110: 64-127      6
2/5:
0:      0-31        5
10:     32-63       5
110:    64-127      6
1110:   128-255     7
11110:  256-511     8
111110: 512-1023    9

Let's start with the short one, since it's very simple:

f9 f4 40 => 1111 1001 1111 0100 0100 0000
2/2 Document index: 111110 011111 => 64 + 31 => Document no. 95
2/1 Code count:     0 1           => 1
2/5 Location codes: 0 00100       => 4       => Word no. 4
    padding:        0000

So, in document no. 95, word no. 4 is a '1' which is in the title. Now, the ordering of the documents is provided by the #URLTBL and #URLSTR files. Looking up document no. 95 in there (0-based indexing!), we see the file is internet_account.htm, in which, the first non-markup text is:

<title>Dial-Up Networking: Step 1</title>
0: dial
1: up
2: networking
3: step
4: 1

Now, the next one is a little more complicated. I won't go over it in as much detail, but I'll just break it out quickly. It contains 10 entries:

(111110 011111) ( 0 1) (   0 00110  )             0000
(    10 00    ) ( 0 1) (  10 10111  )             000
(  1110 1010  ) ( 0 1) (  10 10100  )             0000000
(  1110 1101  ) ( 0 1) (1110 0000000)             000
(     0 01    ) ( 0 1) (  10 11100  )             0000
(111110 001101) ( 0 1) ( 110 010100 )             0
(     0 01    ) ( 0 1) (  10 01111  )             0000
( 11110 11011 ) ( 0 1) ( 110 011001 )             000
(   110 011   ) (10 0) ( 110 011001 ) ( 10 00011) 0000000
(    10 00    ) ( 0 1) ( 110 000001 )             0
                                                  00000000 00000000

Parsing those entries, we get:

Picking one at random, say, Document no. 303 with 2 hits, we open up windows_netsetup_netwin.htm, from which I've generated a wordlist containing all of the words in order:

  0: To(TITLE)
  1: set(TITLE)
  2: up(TITLE)
  ..
  ..
 86: client
 87: follow
 88: steps
 89: 1
 90: 3
 ..
122: follow
123: steps
124: 1
125: 3
126: and
 ..

And we can see the word '1' shows up in precisely the 89th and 124th spots.

The CHM compiler extracts words from all the input files (those with .h in their names), converts them to lowercase and stores them in the $FIftiMain file.

Words in the leaf and index nodes are made up of the following characters stored as is: 0x01 (buggy), 0-9, a-z, _, 0xDE, 0xFE. Bytes from the input files are converted and stored as per the table below. Character entity references of the form ▼ are truncated to BYTEs and translated as per the table below. Character entity references of the form & are treated as whitespace, except for the the latin characters, which are converted as per the table below.

These conversons may depend on the system codepage, character set, font and language set in the HHP file, but I'm just guessing here.

There are a few known bugs that are described in the following few paragraphs.

An 0x01 in a word causes the first whitespace character at the end of the word to be included in the word and if the next character is non-whitespace the word is joined to the next word. If the word begins with 0-9 then the word is terminated before the 0x01 and a new word begins at the 0x01. This bug affects the fields in the initial header. For example: abcd0x1efghi-foobar is converted to abcd0x1efghi-foobar. abcd0x1efghi- foobar is converted to abcd0x1efghi- and foobar. 0bcd0x1efghi-foobar is converted to 0bcd and 0x1efghi-foobar. 0bcd0x1efghi- foobar is converted to 0bcd, 0x1efghi- and foobar.

Weird bug where if the word is 16 characters in length then the word is doubled plus the first 7 chars in length.

There is a weird feature that if a word starts with 0-9 then it may contain multiple periods (0x2E = '.') or commas (0x2C = ',') embedded in the word before the non-period, non-comma word terminating character. I think this feature is so that the user can search for version numbers or numbers with a decimal point or thousands separator in them. Note that commas are removed from the word, while periods are not. For example v1.1.234.5......,6 will become v1 and 1.234.5......6.

There is a weird bug involving words ending in single quote (') being forgotten when the same word is also normal and also ending in a period (.).

There are probably many more hidden bugs and features in the word converter (I think its the the ITIR.StdWordBreaker class in ITIRCL.DLL).

The files in the $WWAssociativeLinks and $WWKeywordLinks directories have the same formats. The maximum total length (including parents) of an entry in one of these files is 488 characters (including NT). HHW complains about and refuses to output any that are greater than this length.

The $WWKeywordLinks dir specifies the contents of the Index navigation pane & the $WWAssociativeLinks dir specifies the Alinks.

In CHW files this is named BTREE and in CHI/CHM files it is named BTree.

This file has a 76 byte header, then 2048 byte blocks. First come all the listing blocks, then all the index blocks. This file is similar to the directory entries in the ITSF format, except that the index blocks are at the end instead of interspersed with the listing blocks. All block indices below are zero based. This file forms a tree, with the last (index mostly) block being the root of the tree. If there is more than one level of index blocks then the root block will have two children; the first in the block header and the second in the entry. WARNING: just as in the ITSF directory there can be garbage in the free space, so respect that first WORD and use it. I'm not yet sure how the listing blocks are split up, though it is probably the same as the ITSF directory (space filling).

'L' = DWORD (indexed)
'F' = NUL-terminated string (indexed)
'i' = NUL-terminated string (indexed)
'2' = WORD
'4' = DWORD
'z' = NUL-terminated string
'!' = DWORD count value, count/8 * record
        DWORD filenumber
        DWORD TopicOffset

In CHW files this is named DATA and in CHI/CHM files it is named Data.

This file contains entries that are 13 bytes in length. All known entries have thus far contained the following bytes: 00000000 05000000 80000000 00. AFAICS this file is useless.

In CHW files this is named MAP and in CHI/CHM files it is named Map.

Begins with a WORD indicating the number of entries in the file (also the number of listing blocks in the BTree file). Each entry is 2 DWORDs. The first is a cumulative sum of the number of keywords in the BTree listing blocks & the second is a consecutively increasing index number. Both start at zero.

In CHW files this is named PROPERTY and in CHI/CHM files it is named Property.

If there are no links of this type in the CHM then this will be a zero DWORD. Othewise it contains the following DWORDs: 0, 0, 0, 0xC, 1, 1, 0, 0. AFAICS this file is pretty much useless.