Tutorials to .com

Tutorials to .com » Software » Delphi » A super-fast reading of the FileStream!

A super-fast reading of the FileStream!

Print View , by: iSee ,Total views: 69 ,Word Count: 1101 ,Date: Mon, 24 Aug 2009 Time: 10:09 AM

Has recently been produced albums for their own software (http://www.tonixsoft.com/ultraalbum/index.php?lang=chs) to open big file slow and depressed when I used to practice first with TFileStream to open a file, Then find the one in which the data segment to which copy the contents to a TMemoryStream, the reason again to copy it to an independent TMemoryStream is because dealing with the aftermath of a file-based database component must accept a whole TStream, as part of its storage the media, the whole process is simply slow intolerable.

The reason is slow, there are two reasons:
1. With TFileStream open the file, operating system, open the file after the document is generated memory mirroring, document a large, open space, and memory copy if the work would become extremely slow.
2. TFileStream part of the re-replication to TMemoryStream, the replication process will open up a new memory re-replication, of course, big memory, copy time become longer.

I am determined to solve the current problems I have, write a file to read class, now called TFastFileStream it, it must be inherited from a TStream, so that other components can be easily combined.

First of all, to be solved is to open large files slow the problem, for this, the use of MapViewOfFile (), will be the file directly as a memory mirroring to access can be had, on the MapViewOfFile (), as well as file memory mirroring, you can refer to this article: http://www.vccode.com/file_show.php?id=2409

delphi established under the file mirroring methods are:
constructor TFastFileStream.Create (const AFileName: String);
var
FileSizeHigh: LongWord;
begin
FFileHandle: = CreateFile (PChar (AFileName), GENERIC_READ, FILE_SHARE_READ, nil, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);
if FFileHandle = INVALID_HANDLE_VALUE then begin
raise FastFileStreamException.Create ( 'Error when open file');
end;
FSize: = GetFileSize (FFileHandle, @ FileSizeHigh);
if FSize = INVALID_FILE_SIZE then begin
raise FastFileStreamException.Create ( 'Error when get file size');
end;
FMappingHandle: = CreateFileMapping (FFileHandle, nil, PAGE_READONLY, 0,0, nil);
if FMappingHandle = 0 then begin
raise FastFileStreamException.Create ( 'Error when mapping file');
end;
FMemory: = MapViewOfFile (FMappingHandle, FILE_MAP_READ, 0,0,0);
if FMemory = nil then begin
raise FastFileStreamException.Create ( 'Error when map view of file');
end;
end;

Finally, the mirror was made of the data stored in the FMemory in, and then cover the TStream the Read () method, when the external need to obtain the data, fetch it into the FMemory in:
function TFastFileStream.Read (var Buffer; Count: LongInt): LongInt;
begin
if (FPosition> = 0) and (Count> = 0) then
begin
Result: = FSize - FPosition;
if Result> 0 then
begin
if Result> Count then Result: = Count;
/ / Move (Pointer (Longint (FMemory) + FPosition) ^, Buffer, Result);
CopyMemory (Pointer (@ Buffer), Pointer (LongInt (FMemory) + FPosition), Result);
Inc (FPosition, Result);
Exit;
end;
end;
Result: = 0;
end;
This function is mainly to imitate TCustomMemoryStream way to write in the same name, but a little strange at first, when I use Delphi's own memory copy function Move (), the program will always have access to the illegal addresses, so we can only be changed with the API Function CopyMemory () has.

In addition, the need to achieve the function are:
function TFastFileStream.GetSize (): Int64;
begin
result: = FSize;
end;

function TFastFileStream.Seek (const Offset: Int64; Origin: TSeekOrigin): Int64;
begin
case Ord (Origin) of
soFromBeginning: FPosition: = Offset;
soFromCurrent: Inc (FPosition, Offset);
soFromEnd: FPosition: = FSize + Offset;
end;
Result: = FPosition;
end;
In this way, a complete set of documents will have a reading mechanism.

As the complexity of the relationship, I did not realize the file save mechanism, a friend invited interested in achieving his bar.

Next, the need to address is how to use the two current operators to optimize the Stream replication to start thinking approach is to create a new Stream class Stream copying it out from other data, is not a new open memory Instead, the internal memory pointer to data blocks within the source Stream in a certain period, but such a result, the Stream class in the source Stream only be used during the lifetime of the relationship seemed to become confused.

Later, suddenly thought of another way, in fact, is for the outer class (which I have used file-based database component), it is only using the Read (), Seek () and other methods to access the data, then I just use some deceptive means to enable the internal class only to return to the outside of their internal data in a certain period on it.
For my program run, I would like to find the location of data, its set up a virtual range of data, external access in the future, they all return to the virtual data within the data. As a result, only the original TFastFileStream based on a certain degree of transformation on it.

procedure TFastFileStream.SetUseableMemory (const StartPos: Int64; const Size: Int64);
begin
FUseableStartPos: = StartPos;
FSize: = Size;
end;

function TFastFileStream.Read (var Buffer; Count: LongInt): LongInt;
begin
...
CopyMemory (Pointer (@ Buffer), Pointer (LongInt (FMemory) + FUseableStartPos + FPosition), Result);
...
end;

Well, stop here on the end of the transformation, and finally put on the FileStream to write this new class, a test, the speed is amazingindeed, the original open a nearly 30MB of the file, use the 2 Stream class, take about 40 seconds and change into a new TFastFileStream, it only needs a class on getting out of time in 5 seconds or less, ha ha, it really cool Ah!

If you need the complete code for this class, you can write contact me:
tonyki [at] citiz.net


Delphi Tutorial Articles


Can't Find What You're Looking For?


Rating: 2.0

Comments

No comments posted.