Xbase DBMS Chapter 2

From c894a7cdd8686ea695602a23a511a3f1b0d047be Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B6rg=20Frings-F=C3=BCrst?= Date: Mon, 14 Aug 2023 21:07:46 +0200 Subject: New upstream version 4.1.4 --- docs/html/xbc2.html | 368 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 368 insertions(+) create mode 100755 docs/html/xbc2.html (limited to 'docs/html/xbc2.html') diff --git a/docs/html/xbc2.html b/docs/html/xbc2.html new file mode 100755 index 0000000..71d4585 --- /dev/null +++ b/docs/html/xbc2.html @@ -0,0 +1,368 @@ + + +Xbase DBMS Chapter 2 + +

System Overview

Chapter Updated 04/28/23

+ +

+The XBase64 library is a cross platform object oriented C++ set of classes for +accessing and manipulating the following Xbase file types:

+ + + + + + + + + + + + +

Extension	Description	Supported by Xbase64 library?
*.DBF	Main DBF file or table name	Y
*.DBT	Verson 3 or Version 4 memo file	Y
*.NDX	Single tag index file	Y
*.MDX	Multi tag index file	Y
*.INF	ODBC file used for auto open of NDX files	Y
*.NTX	Clipper indexfile	Future
*.CDX	Fox Pro indexfile	Future
*.IDX	Fox Pro index file	Future

+ + + + +

+ +DBF files are comprised of a variable length header record which stores +information about the file and describes he fixed length record format, +followed by a series of fixed length data records. +

+ +Each fixed length data record is preceded by a one byte indicator +which identifies if the record has been deleted. If the record is +not deleted, the indicator is a space (0x20). If deleted, the +indicator contains an asterisk (0x2A). Data fields are stored in records +without field separators or record terminators.

+ +In earlier releases of dBASE, there is an ASCII NULL character +between the $0D end of header indicator and the start of the data. +This NULL was removed starting with dBASE III Plus, making a Plus +header one byte shorter than an identically structured III file. +The methods documented in the Xbase software and documentation follow +the more recent version where the NULL character is not included. +

+ +Each database file is comprised of zero, one or many records. A record is +comprised of fields. Only one record is accessed at a time.

+ +Zero, one or many database files can be open simultaneously.

+ +

+ +

The Record Buffer

+ +When using the Xbase routines, each open data file has a record buffer +which is manipulated by calling the database, index and field routines. +

+ +If AutoCommit is turned on (Default), updates are committed from +the record buffer to the database when a write, or append is performed. +The library automatically writes updates to the database if the buffer has +been updated and the record is repositioned or the database is closed. +

+ +If AutoCommit is turned off, updates will need to be explicity +committed to the database file with one of dbf->Put(), dbf->Append() +or dbf->Commit() command depending on context.. +Updates can be cancelled with the Abort() command. +

+The record buffer is not used for handling the actual data portion of +memo fields. When working with memo fields, the application program must +allocate enough buffer space for reading and writing memo fields or use +the xbString class for handling memo data.

+ +Internal to the library, there is an additional record buffer which +stores the original value of the data record before any changes are made. +This is used by the index routines for finding and deleting original key +values from any open indices before adding the new keys. If the key values +are not changed, no index updates occur. Additionally, calling the Abort() +method will back out any updates to the record buffer. + + +

+

+
+

Xbase Database File Header - DBF Version III and Version IV

+ +The Xbase file header, located at the beginning of the database, describes +the .DBF database. Knowledge of this structure is not necessary to +effectively utilize the Xbase64 libraries.

+ + + +

Position	Length	Description +
0	1 byte	file version number + (03H without a .DBT file) + (83H with a .DBT file) +
1-3	3 bytes	date of last update + (YY MM DD) in binary format +
4-7	32 bit number	number of records in data file +
8-9	16 bit number	length of header structure +
10-11	16 bit number	length of the record +
12-31	20 bytes	reserved +
32-n	32 bytes each	field descriptor record (see below) +
n+1	1 byte	0DH as the field terminator +

+

+ +

+
+

Xbase Field Descriptor Record

+The Xbase field descriptor record stores information about each field in the +database. Each database has from 1 to 1024 fields. +Knowledge of this structure is not necessary to +effectively utilize the Xbase libraries.

+ + + +

Position	Length	Description +
0-10	11 bytes	field name in ASCII zero-filled +
11	1 byte	field type in ASCII (C N L D or M) +
12-15	32 bit number	field data address +
16	1 byte	field length in binary +
17	1 byte	field decimal count in binary +
18-31	14 bytes	reserved bytes (version 1.00) +

+

+

+
+

Field Data Format

+Data are stored in ASCII format in the database as follows:

+ + +

DATA TYPE	DATA RECORD STORAGE +
Character	ASCII characters, left justified, right blank filled +
Date	(8 digits in YYYYMMDD format, such as + 19601007 for October 7, 1960) +
Logical	? Y y N n T t F f (? when not initialized) +
Memo	10 digits representing a .DBT block number +
Numeric	. 0 1 2 3 4 5 6 7 8 9 + -, right justified, left blank filled +
Float (Version IV only)	. 0 1 2 3 4 5 6 7 8 9 + -, right justified, left blank filled +

+

+ +

Memo Fields

+ +Memo fields store variable length data elements in a seperate .DBT file. +The main .DBF file maintains a ten byte field which is used by the Xbase +routines for determining the location of the data in the .DBT file. +

+ +Xbase DBMS supports both dBASE III+ and dBASE IV version memo files. +The version IV files are somewhat more efficient in that they reuse +unused memo space when data are deleted or freed from use. With version +III files, all new updates are appended to the end of the file and the +unused space is not reclaimed until the datafiles are packed. +

+ +Memo fields can be used for storing a variety of date type. However, +type 3 files are limited to storing textual data because most internal +memo field processing in a type 3 file relies on two contiguous 0x1a +charaters.

+ +Type 4 memo fields can be used for storing BLOB (binary large object) +data reliably, as the internal file structure does not rely on any +special characters embedded in the data.

+ + +

Technical memo file information

+ +The following info on memo fields is for the curious. +It is not required +reading if you don't need to know the internals.

+ +

Memo files are made up of one or more blocks +

For version III files, the block size is 512 +

For version IV files, the block size is a multiple of 512 +

The minimum amout of space necessary to store one memo field is +one block or 512 bytes. +

The default block size can be adjusted by manipulating the +XB_DBT_BLOCK_SIZE macro in the options.h file. + + +

The main .DBF file maintains a ten byte numeric field which is blank if +no memo data exists for a given field. Otherwise it contains a number, which +when multiplied by the block size, points to the offset in the file of the head +block in the file/ +

+ +For version 3 memo field files, there are two fields in the head block of +the file, NextBlockNo and Version. Depending on the +Xbase software, some vendors products update these two fields, some do not. +The Xbase library keeps the fields updated, but does not rely on them to +be valued with correct data. This helps to support maximum compatibility +amoungst all Xbase tools available.

+ +For version 4 memo field files, +the first block in the .DBT file is a header block which is comprised of +8 bytes of data which maintain the file's block size and the next free +block available in the file. Blocks two through n contain the actual +memo data. A chain of empty blocks is maintained within the file for +potential future use. When an add or update routine executes, it first +attempts to find a spot in a set of blocks which were earlier allocated, +but not currently in use for the data. If no free spot is found, data are +appended to the end of the file. + +The free block chain is sorted in block number order. When blocks of +data are freed and added to the free block chain, the routines will attempt +to concatonate free block chains togethor where possible. When a delete +occurs, or an update which requires less space occurs, the new free space +is added to the free block chain. + +

+ +

Various Memo File Block Types

+ + + +

Valid Block Types +
Head Block +
Only data block for memo field +
First of several contiguous data block set +
2-n of contiguous data block set +
Only data block in free chain (version IV only) +
First of several contiguous free block set (version IV only) +
2-n of contiguous free block set (type 4 only) +

+

+ +

Head Block Structure

+ + +

1-4	LONG	Next Block ID +
5-8	LONG	Not used all 0x00's +
9-16	CHAR(8)	Filename (Version IV Only) +
17	CHAR	Version (0x03 = Version III, 0x00 = Version IV) +
18-20	CHAR(3)	Not used all 0x00's +
21-22	SHORT	Block Size (Version IV only ) +
23-Remainder of block	CHAR	Not used +

+

+ + +

Version IV Head Data Block Structure

+ + +

xbShort	0-1	-1 +
xbShort	2-3	Starting position of data (always 8 ?) +
xbLong	4-7	Length of data includes first 8 bytes +
char (9) - Blocksize	8-15	Data +

+

+ +

Version IV Head Free Block Structure

+ + +

xbLong	0-3	Next free block in the free block chain +
xbLong	4-7	Number of free blocks in this contiguous free + block set +

+

+Version 3 and 4 memo fields are terminated with two contiguous 0x1A bytes of data. +

+

+ + +

Block Reads

+ +As of release 4.1.1, the Xbase library includes functionality for reading a DBF file in blocks, rather than one record at a time.

+ +This functionality can be used to improve application performance in situations where a data file is being read sequentially. +For situations where records are retrieved randomly from the file, enabling this probably won't help much. In short, this +can be turned on when accessing a file sequentially and should be left off when not processing sequentially.

+ +The logic is handled internally within the library, all that is needed is to +enable it and the library handles the rest. Additionally, the block read functionality +was designed with for sequential file access and is designed with reporting in mind. +It doesn't currently have any auto locking associated with it. +

+ +To enable and disable Block Reading for a DBF file, use +xbDbf::EnableBlockReadProcessing() +and xbDbf::DisableBlockReadProcessing(). +

+ +

Locking Overview

+ +Xbase64 supports multi-user processing through file and record locks. +Record locking restricts multiple cooperating programs from simultaneously +accessing the same data and corrupting it. Without record and file locking +in a multi-user environment, simultaneous access to the data and index files +can cause the files to become inaccurate and unusable.

+ +Automatic record locking is on by default in the Xbase64 library. To disable it, +use method xbXBase::DisableDefaultAutoLock() and to enable it, use method xbXBase::EnableDefaultAutoLock(). +

+Locking can also be enabled / disabled at the table level with with xbDbf::SetAutoLock().

+If autolocking is disabled and the code base is being used in a multi user environment, it is +up to the application program to verify the needed locks are set as there is no checking or +setting any locks if autolocking is turned off. It is only safe to turn off the autolocking functionality +if the library is being used in a single user environment. + +

+The current Xbase64 record locking logic is modeled after DBase (tm) V7 locking. +

+ +The locking methods return either XB_LOCK_FAILED or XB_NO_ERROR. If they return +XB_LOCK_FAILED the actual reason can be found in the global variable +errno or function perror() can be executed to view the +results. +

+ +The errno field may contain one of the following values if the lock was not +successful.

+ + +

Error Code	Description +
EBADF	Invalid file descriptor +
EINVAL	Invalid lock information or file does not support locks +
EACCESS EAGAIN	Lock can not be set because it is blocked by an existing lock on the file. +
ENOLCK	The system is out of lock resources, too many file locks in place. +
EDEADLK	Deadlock condition +
EINTR	Process was interrupted by a signal while it was waiting +

+

+ +

Linux/Windows File Locking Compatibility Issue

+ +There is a compatibility locking issue to be aware of. Windows environments allow for the exclusive +opening of file handles and Linux/Unix platforms do not. If you are writing an application that will be +using a tool like Dbase on a Windows machine, accessing a file on a Linux/Samba configure machine, +be aware that the file could be opened in exclusive mode by DBase on the Windows system, and the same file could +be simultaneously opened with a program on the Unix box. That could cause some issues. + +

+In Unix, a program can not lock a file so another process can not access it.
+In Windows, a program can lock a file so another process can not access it.
+DBase(tm) supports routines to open files exclusively, preventing other users from opening a file.
+Locking on the Mac/Apple platform only works on NFS shares. It does not work with SMB shares. + +

Samba settings

+ +If you will be using Samba on Linux/Unix and sharing files between Linux and Windows machines, +you will need to disable oplocks. In the smb.conf file, set:
+
+[sharename]
+oplocks = False
+level2 oplocks = False + + +

iLockFlavor

+ +The library was constructed in a manner so that it could be updated to support alternate lock "flavors". +The 4.x.x library is built to mirror the DBase locking, but the structure is in place to expand to other locking +types if needed. + + +

+

+ + + -- cgit v1.2.3