1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
|
<!DOCTYPE HTML PUBLIC>
<HTML>
<TITLE>Xbase DBMS Chapter 2</TITLE>
<BODY BGCOLOR=#FFFFFF>
<H2><p align="center">Database Overview</p></H2>
<p align="center">Chapter Updated 11/21/22</p><hr>
The objective of this chapter is to provide information regarding how
the database files are utilized and document the various record structures.
With the exception of the brief section on the record buffer, the
information presented in this chapter is not required to use the
Xbase library. It is mainly information describing internal file
structures utilized by the Xbase routines.<br><br>
Xbase DBF files are comprised of a variable length header record which stores
information about the file and describes
the fixed length record format, followed by a series of fixed length
data records.
<br><br>
Each fixed length data record is preceded by a one byte indicator
which identifiies if the record has been deleted. If the record is
not deleted, the indicator is a space (0x20). If deleted, the
indicator contains an asterisk (0x2A). Data fields are stored in records
without field separators or record terminators.<br><br>
In earlier releases of dBASE, there is an ASCII NULL character
between the $0D end of header indicator and the start of the data.
This NULL was removed starting with dBASE III Plus, making a Plus
header one byte shorter than an identically structured III file.
The methods documented in the Xbase software and documentation follow
the more recent version where the NULL character is not included.
<br><br>
Each database file is comprised of zero, one or many records. A record is
comprised of fields. Only one record is accessed at a time.<br><br>
Zero, one or many database files can be open simultaneously.<br><br>
<hr>
<h3>The Record Buffer</h3>
When using the Xbase routines, each open data file has a record buffer
which is manipulated by calling the database, index and field routines.
<br><br>
If <i>AutoCommit</i> is turned on (Default), updates are committed from
the record buffer to the database when a write, or append is performed.
The library automatically writes updates to the database if the buffer has
been updated and the record is repositioned or the database is closed.
<br><br>
If <i>AutoCommit</i> is turned off, updates will need to be explicity
committed to the database file with one of <i>dbf->Put(), dbf->Append()
or dbf->Commit()</i> command depending on context..
Updates can be cancelled with the <i>Abort()</i> command.
<br><br>
The record buffer is not used for handling the actual data portion of
memo fields. When working with memo fields, the application program must
allocate enough buffer space for reading and writing memo fields or use
the xbString class for handling memo data.<br><br>
Internal to the library, there is an additional record buffer which
stores the original value of the data record before any changes are made.
This is used by the index routines for finding and deleting original key
values from any open indices before adding the new keys. If the key values
are not changed, no index updates occur. Additionally, calling the Abort()
method will back out any updates to the record buffer.
<br><br>
<hr>
<br>
<h3>Xbase Database File Header - DBF Version III and Version IV</h3>
The Xbase file header, located at the beginning of the database, describes
the .DBF database. Knowledge of this structure is not necessary to
effectively utilize the Xbase64 libraries.<br><br><br>
<TABLE BORDER>
<TR valign="BASELINE">
<TR><TH ALIGN="LEFT">Position<TD>Length<TD>Description
<TR><TH ALIGN="LEFT">0<TD>1 byte<TD>file version number<br>
(03H without a .DBT file)<br>
(83H with a .DBT file)
<TR><TH ALIGN="LEFT">1-3<TD>3 bytes<TD>date of last update<br>
(YY MM DD) in binary format
<TR><TH ALIGN="LEFT">4-7<TD>32 bit number<TD>number of records in data file
<TR><TH ALIGN="LEFT">8-9<TD>16 bit number<TD>length of header structure
<TR><TH ALIGN="LEFT">10-11<TD>16 bit number<TD>length of the record
<TR><TH ALIGN="LEFT">12-31<TD>20 bytes<TD>reserved
<TR><TH ALIGN="LEFT">32-n<TD>32 bytes each<TD>field descriptor record (see below)
<TR><TH ALIGN="LEFT">n+1<TD>1 byte<TD>0DH as the field terminator
</TABLE>
<BR><BR>
<hr>
<br>
<h3>Xbase Field Descriptor Record</h3>
The Xbase field descriptor record stores information about each field in the
database. Each database has from 1 to 1024 fields.
Knowledge of this structure is not necessary to
effectively utilize the Xbase libraries.<br><br><br>
<TABLE BORDER>
<TR VALIGN="BASELIGN">
<TR><TH ALIGN="LEFT">Position<TD>Length<TD>Description
<TR><TH ALIGN="LEFT">0-10<TD>11 bytes<TD>field name in ASCII zero-filled
<TR><TH ALIGN="LEFT">11<TD>1 byte<TD>field type in ASCII (C N L D or M)
<TR><TH ALIGN="LEFT">12-15<TD>32 bit number<TD>field data address
<TR><TH ALIGN="LEFT">16<TD>1 byte<TD>field length in binary
<TR><TH ALIGN="LEFT">17<TD>1 byte<TD>field decimal count in binary
<TR><TH ALIGN="LEFT">18-31<TD>14 bytes<TD>reserved bytes (version 1.00)
</TABLE>
<BR><BR>
<hr>
<br>
<h3>Field Data Format</h3>
Data are stored in ASCII format in the database as follows:<br><br>
<TABLE BORDER>
<TR VALIGN="BASELIGN">
<TR><TH ALIGN="LEFT">DATA TYPE<TD>DATA RECORD STORAGE
<TR><TH ALIGN="LEFT">Character<TD>ASCII characters, left justified, right blank filled
<TR><TH ALIGN="LEFT">Date<TD>(8 digits in YYYYMMDD format, such as<BR>
19601007 for October 7, 1960)
<TR><TH ALIGN="LEFT">Logical<TD>? Y y N n T t F f (? when not initialized)
<TR><TH ALIGN="LEFT">Memo<TD>10 digits representing a .DBT block number
<TR><TH ALIGN="LEFT">Numeric<TD>. 0 1 2 3 4 5 6 7 8 9 + -, right justified, left blank filled
<TR><TH ALIGN="LEFT">Float (Version IV only)<TD>. 0 1 2 3 4 5 6 7 8 9 + -, right justified, left blank filled
</TABLE>
<BR><BR>
<hr>
<h3>Memo Fields</h3>
Memo fields store variable length data elements in a seperate .DBT file.
The main .DBF file maintains a ten byte field which is used by the Xbase
routines for determining the location of the data in the .DBT file.
<br><br>
Xbase DBMS supports both dBASE III+ and dBASE IV version memo files.
The version IV files are somewhat more efficient in that they reuse
unused memo space when data are deleted or freed from use. With version
III files, all new updates are appended to the end of the file and the
unused space is not reclaimed until the datafiles are packed.
<br><br>
Memo fields can be used for storing a variety of date type. However,
type 3 files are limited to storing textual data because most internal
memo field processing in a type 3 file relies on two contiguous 0x1a
charaters. <br><br>
Type 4 memo fields can be used for storing BLOB (binary large object)
data reliably, as the internal file structure does not rely on any
special characters embedded in the data.<br><br>
A special note on storing string data in a memo field. For those users
that are new to C/C++ programming, string fields typically end with
a null (0x00) terminator character. As a general rule of thumb when using
the library, add one to the length of any string when
specifying the length of the data. This stores the null terminating byte
with the data. For example, when storing string "This is a string"
specified size should be 17, not 16.
<h4>Technical memo file information</h4>
The following info on memo fields is for the curious.
It is not required
reading if you don't need to know the internals.<br><br>
<li>Memo files are made up of one or more blocks
<li>For version III files, the block size is 512
<li>For version IV files, the block size is a multiple of 512
<li>The minimum amout of space necessary to store one memo field is
one block or 512 bytes.
<li>The default block size can be adjusted by manipulating the
XB_DBT_BLOCK_SIZE macro in the options.h file.
<li>The main .DBF file maintains a ten byte numeric field which is blank if
no memo data exists for a given field. Otherwise it contains a number, which
when multiplied by the block size, points to the offset in the file of the head
block in the file/
<br><br>
For version 3 memo field files, there are two fields in the head block of
the file, <em>NextBlockNo</em> and <em>Version</em>. Depending on the
Xbase software, some vendors products update these two fields, some do not.
The Xbase library keeps the fields updated, but does not rely on them to
be valued with correct data. This helps to support maximum compatibility
amoungst all Xbase tools available.<br><br>
For version 4 memo field files,
the first block in the .DBT file is a header block which is comprised of
8 bytes of data which maintain the file's block size and the next free
block available in the file. Blocks two through n contain the actual
memo data. A chain of empty blocks is maintained within the file for
potential future use. When an add or update routine executes, it first
attempts to find a spot in a set of blocks which were earlier allocated,
but not currently in use for the data. If no free spot is found, data are
appended to the end of the file.
The free block chain is sorted in block number order. When blocks of
data are freed and added to the free block chain, the routines will attempt
to concatonate free block chains togethor where possible. When a delete
occurs, or an update which requires less space occurs, the new free space
is added to the free block chain.
<br><br>
<h3>Various Memo File Block Types</h3>
<TABLE BORDER>
<TR VALIGN="BASELIGN">
<TR><TH ALIGN="LEFT">Valid Block Types
<TR><TH ALIGN="LEFT">Head Block
<TR><TH ALIGN="LEFT">Only data block for memo field
<TR><TH ALIGN="LEFT">First of several contiguous data block set
<TR><TH ALIGN="LEFT">2-n of contiguous data block set
<TR><TH ALIGN="LEFT">Only data block in free chain (version IV only)
<TR><TH ALIGN="LEFT">First of several contiguous free block set (version IV only)
<TR><TH ALIGN="LEFT">2-n of contiguous free block set (type 4 only)
</TABLE>
<BR><BR>
<h3>Head Block Structure</h3>
<TABLE BORDER>
<TR VALIGN="BASELIGN">
<TR><TH ALIGN="LEFT">1-4<TD>LONG<TD>Next Block ID
<TR><TH ALIGN="LEFT">5-8<TD>LONG<TD>Not used all 0x00's
<TR><TH ALIGN="LEFT">9-16<TD>CHAR(8)<TD>Filename (Version IV Only)
<TR><TH ALIGN="LEFT">17<TD>CHAR<TD>Version (0x03 = Version III, 0x00 = Version IV)
<TR><TH ALIGN="LEFT">18-20<TD>CHAR(3)<TD>Not used all 0x00's
<TR><TH ALIGN="LEFT">21-22<TD>SHORT<TD>Block Size (Version IV only )
<TR><TH ALIGN="LEFT">23-Remainder of block<TD>CHAR<TD>Not used
</TABLE>
<BR><BR>
<h3>Version IV Head Data Block Structure</h3>
<TABLE BORDER>
<TR VALIGN="BASELIGN">
<TR><TH ALIGN="LEFT">xbShort<TD>0-1<TD>-1
<TR><TH ALIGN="LEFT">xbShort<TD>2-3<TD>Starting position of data (always 8 ?)
<TR><TH ALIGN="LEFT">xbLong<TD>4-7<TD>Length of data includes first 8 bytes
<TR><TH ALIGN="LEFT">char (9) - Blocksize<TD>8-15<TD>Data
</TABLE>
<BR><BR>
<h3>Version IV Head Free Block Structure</h3>
<TABLE BORDER>
<TR VALIGN="BASELIGN">
<TR><TH ALIGN="LEFT">xbLong<TD>0-3<TD>Next free block in the free block chain
<TR><TH ALIGN="LEFT">xbLong<TD>4-7<TD>Number of free blocks in this contiguous free
block set
</table>
<br><br>
Version 3 and 4 memo fields are terminated with two contiguous 0x1A bytes of data.
<br><br>
<hr>
<p><img src="xbase.jpg"><hr>
</BODY>
</HTML>
|
