<< New in Firebird 2.5 | Firebird 2.5.3 Release Notes | Changes to the Firebird API and ODS >>

Changes in the Firebird engine

The primary objective of this release was to refactor Firebird's threading architecture to take advantage of the symmetric multiprocessing (SMP) capabilities of multiprocessor hardware. This has a noticeable effect on the scaleability of Superserver when multiple databases are being accessed simultaneously but its major effect is the emergence of the architectural "Superclassic" model that will underpin the fine-grained multi-threading under development for Firebird 3.

New threading architecture

Dmitry Yemanov
Vladyslav Khorsun
Alex Peshkov
also -
Nickolay Samofatov
Roman Simakov

For Superserver, the new architecture will be most obvious in two ways:

1. In a multiple database environment, Superserver threads for each database are allotted evenly to available processors.

2. A slight improvement in scaling should be apparent for single database usage on SMP hardware.

It is with Classic that the effects are most evident:

1. Classic Server can now be multi-threaded. The one worker thread per process model remains but now it is possible to use additional threads for parallel tasks such as asynchronous shutdown, sweep, inter-process communications with the lock manager and more.
2. On POSIX, services in Classic also run in threads now, rather than in forked processes as previously.

3. The embedded libraries — libfbembed.so on POSIX and fbembed.dll on Windows—are now multi-thread-capable and thread-safe, so they can be used in multi-threaded applications.
4. Testing suggests that the performance of Classic in this version will be be significantly faster than previous Classic versions.

back to top of page

"Superclassic"

This multi-threaded mode for Classic has been dubbed "Superclassic" for its capability to handle multiple worker threads - dedicated or pooled - inside a single server process. It shares all the usual Classic features, with a few differences:

• Safe, full shutdown of the server engine is possible on any platform.
• Under some TPC conditions, it can outperform Classic - by about 15-20%.
• It uses fewer kernel resources (although not less memory).
• When a Superclassic process crashes, it takes all its connections with it.
• Recognised limitations in the Services API for the Classic server, such as the inability to retrieve the list of attachments/active users, do not apply to Superclassic.
• On POSIX, Superclassic does not require [x]inetd.

Embedded Server Notes

• The embedded server in the Windows library, fbembed.dll, now uses Superclassic, not Superserver as previously, thus unifying its model with that of local connection to Superclassic on POSIX. The database file lock that previously restricted connections to a single application space is replaced by a global lock table that allows simultaneous access to the same database from different embedded server modules. This facilitates concurrent debugging of applications and use of native utility tools like gbak, gstat and so on.
• A single attachment handle can now be shared by simultaneous threads. (Tracker reference CORE-2498, A. dos Santos Fernandes).

back to top of page

Usage notes

Windows

On Windows, the same fb_inet_server.exe binary delivers either the Classic or the Superclassic working modes, according to switch settings. Classic is the default mode.

To use the Superclassic mode as a service, add the -m[ulti-threaded] switch to the instsvc.exe command line, as follows:

 instsvc install -multithreaded

When intending to run Superclassic as an application, use

 fb_inet_server -a -m

New binary for POSIX

On POSIX, the new binary fb_smp_server is supplied for the Superclassic model. It contains the network listener, meaning it works similarly to fbserver with regard to attachment requests and does not require [x]inetd.

The multi-threaded engine used by fb_smp_server is the libfbembed.so library, in accordance with OSRI requirements. The Classic packages also include fbguard (the Guardian) which, in a Superclassic installation, starts fb_smp_server, rather than fbserver as it does when the Superserver model is installed with Guardian.

Important: Do not try to use fbguard when running the traditional Classic server.

back to top of page

Thread-safe client library

Dmitry Yemanov
Vladyslav Khorsun
Alex Peshkov

Tracker reference CORE-707.

The client libraries, including the embedded one, can now be used in multi-threaded applications without any application-level synchronization.

Improvements

Improvements implemented include:

Immediate detection of disconnected clients on Classic

Vladyslav Khorsun

The Classic server now detects immediately when a Classic process has been broken by a client disconnection. Its response is to terminate any pending activity, roll back the active transaction and close the network connection.

Tracker reference CORE-818.

Optimizations

Important optimizations include:

Data Retrieval

Dmitry Yemanov

An optimization improves data retrieval performance for tables from which no fields are accessed. This applies, for example to the SELECT COUNT(*) style of query.

Tracker reference CORE-1598.

BLOB memory usage

Adriano dos Santos Fernandes

An optimization avoids memory consumption of <page size> bytes for each temporary BLOB created during assignment.

Tracker reference CORE-1658.

Performance improvement for updates

V. Khorsun

The aim of this improvement was to reduce the amount of precedence writing that the engine undertakes when performing its "careful write" procedure for updates. The existing procedure had a noticeable effect on the performance of mass updates, especially for "updates-in-place", where the same records are updated more than once in the same transaction. In the worst case, a page could be written to disk for every single new record version created during an update.

See the Tracker reference (CORE-2672) for a simplified technical description of what was involved internally.

The solution addresses the potentially time-consuming process involved in protecting the write operations from causing circular references between the pages on which new record versions and back versions are placed to maintain correct precedence.

back to top of page

Cache size limit increased for 64-bit servers

V. Khorsun

Tracker reference CORE-1687

Previous 64-bit Firebird server versions could not benefit from 64-bit address space and be configured for more than 2 GB (16K * 128 K)of database cache. The problem has been rectified in this version. On 64-bit Firebird, if the resources are available, it is now possible to configure cache large enough to accommodate a database of 5-10 GB completely in RAM.

Although Firebird's caching can get a lot of help from the file system cache, it is a feature that could be important for high-throughput systems whose load is mainly reads. The theoretical upper limit for caches on x64 Firebird servers is now 2^31 -1 (2,147,483,647) pages.

Default database location

A. Peshkov

Tracker reference CORE-1643

The configuration parameter DatabaseAccess now has more "meaning" attached to it. In the absence of any other indication, the first location defined in the Restrict list for DatabaseAccess is taken by the engine as the default location for creating a new database and for locating a database where the connection parameters do not specify either an alias or the full path specification.

The seek logic is similar to that use for finding external tables from the Restrict list supplied to the ExternalFileAccess parameter, viz.,

1. All directories in the Restrict list are searched first.
2. If the database specified is not found:

• If CREATE DATABASE is involved, then the first location in the Restrict list is used.
• Otherwise, the attach fails in the expected fashion.

Note: Current Working Directory

This feature does not suppress the use of the current working directory as the implicit location of the specified database file for direct local connections. The Y-valve handles the path resolution in these cases, just as it always did.

For a stand-alone server working via the remote subsystem, trying to connect using the database file name with no path, although unlikely, is not recommended, since there is not really any way to be certain where the database would "land". On Windows, for example, under these conditions the current working directory would be %system%.

back to top of page

DLL loading for Windows embedded engine

Adriano dos Santos Fernandes

The root determination mechanism for the Windows embedded engine has been changed to avoid common problems that occur when an installation of the application structure encounters "DLL Hell". Previously, the implicit root directory was the directory containing the user application's main executable file. Now it is the directory where the renamed fbembed.dll library is located.

Tracker reference CORE-1814.

Large external table support enabled

Vlad Khorsun

Previous Firebird versions used 32-bit I/O when working with external tables, limiting the size of the external file to < 2 GB. The mechanism has been enhanced to use 64-bit I/O on file systems that support it, effectively eliminating the 2 GB limit.

Tracker reference CORE-2492.

Statistics now work properly with 64-bit values

V. Khorsun
A. Peshkov

Tracker reference CORE-2619

Memory and other statistics did not work properly 64-bit values in older Firebird versions. The issue had two parts:

a. To make the engine use 64-bit integers for statistics, the internals of AtomicCounter were changed.
b. The isql and qli tools had to be taught to work with 64-bit values.

Incompatibility with older clients: To enable the 32-bit tools to work correctly with a 64-bit server, it was necessary to introduce some new internal API functions (struct perf64 and perf64_xxx) and change isql and qli to use them. This means that the isql and qli programs in v.2.5 are not compatible with older Firebird clients.

UDFs safeguard

Adriano dos Santos Fernandes

Tracker reference CORE-1937.

When a string UDF is written to return a pointer not allocated by the same runtime as the Firebird server is accessing, the presence of the FREE_IT keyword in its declaration corrupts memory and crashes the server. As a safeguard against such dysfunctional UDFs, the engine now

1. detects such UDFs and throws an exception,
2. depends on the presence of the updated ib_util library in the path for all server models, including embedded.

back to top of page

Diagnostics

Transaction diagnostics

Claudio Valderrama

Better diagnostics and error reporting when TPB contents are malformed. The new TPB validation logic now rejects:

• explicitly conflicting options within the same category, e.g., {WAIT} and {NOWAIT} specified together, or {READ COMMITTED} and {SNAPSHOT}, or {READ ONLY} and {WRITE},
• options making no sense, e.g. [NO] RECORD VERSION specified for a SNAPSHOT isolation mode,
• incorrect order of table reservation options, e.g. {PROTECTED READ <TABLE>} instead of {READ <TABLE> PROTECTED}.

Tracker reference CORE-1600.

Access privilege error messages

Alex Peshkov

Both table and column names are now reported when access privilege exceptions occur for a column.

Tracker reference CORE-1234.

Message improvement

V. Khorsun

Tracker reference CORE-2587

The diagnostic message when the engine cannot create shared memory that has already been mapped by another engine process in another Windows session is now a bit more user-friendly. It used to say:

The requested operation cannot be performed on a file with a user-mapped section open.

Now, it says:

Database is probably already opened by another engine instance in another Windows session.

Metadata improvements

Preserve character set default collation

Adriano dos Santos Fernandes

An improvement allows the current value of RDB$DEFAULT_COLLATE_NAME in the system table RDB$CHARACTER_SETS to survive the backup/restore cycle. The mechanism for such customisation is the new ALTER CHARACTER SET command.

Tracker reference CORE-789.

back to top of page
<< New in Firebird 2.5 | Firebird 2.5.3 Release Notes | Changes to the Firebird API and ODS >>