[aplusdev] Mapped files

[Alexei Zalivine]

Hi Jon,

Thank you for good answer. But I try to describe my task.
So I have a object with gauges (about 1,5 hundred). Each second I would like
to register data from its and store to file (or database). If object work
properly, I don't doing anything. But, if gauges register any anomaly, I
must analyze the situation and report the reason to operator. All data must
be stored and has access in any time. So I have 2 basic I/O tasks:
1)The fast write to file (about 150 gauge's values and other addition values
per second ). I test mapped files with float data and give very best
performance.
2)The fast access to these data. I'm very like the solution with mapped
files and variables. If I right understand, the mapped variable do not fully
placed in memory, and I can do search, filtering, and other actions with
these data.

Other words, I want to build simple database on mapped files. But some
arrays, which must be stored has data with mixed type (type `box, like the
example in my first mail). If I will be use sys.exp{} and sys.imp functions,
I can't use such calling as f[,]<-, f[index;] and so on, where f is mapped
variable.

May be I can use the dependencies or other A+ techniques?

Tnaks a lot,
Alexei.


>
> Hi, Alexei.  I would have made the same suggestion that John Mizel
> gave to you, in using the "sys.exp" and "sys.imp" functions.  You
> are indicating that the solution isn't what you are after, so let
> me just make sure that we are all thinking of the same thing.
>
> "sys.exp" (export) and "sys.imp" (import) are utility functions
> that can be thought of as being almost a part of the file-write
> and file-read facilities.  Because they are separate, however,
> you can make sure that you aren't taking any extra overhead for
> them in cases where you do *not* need to use them.
>
> Those functions are used for storing and reading nested arrays.
> They work by converting your data into a sort of an encoded form,
> which is then written out in the form of a flat character vector.
> But I want to emphasize that you would not be using that character
> vector yourself; your data in your workspace would continue to be
> nested data... it is only at the moment that the data is written
> to the file that it is converted.
>
> For example:
>
>      mû3
> 4Ò(1;'Lesha';10;'Mandrake';2;'Sasha';15;'SuSE';3;'Dima';5;'RedHat')
>
>      $load disp     ã get fns for displaying data structure
>      disp.disp m    ã "display" the matrix
> +4--------------------------+
> 3+-+ +5----+ +--+ +8-------+|
> ||1| |Lesha| |10| |Mandrake||
> |+i+ +"----+ +i-+ +"-------+|
> |+-+ +5----+ +--+ +4---+    |
> ||2| |Sasha| |15| |SuSE|    |
> |+i+ +"----+ +i-+ +"---+    |
> |+-+ +4---+  +-+  +6-----+  |
> ||3| |Dima|  |5|  |RedHat|  |
> |+i+ +"---+  +i+  +"-----+  |
> +Ì--------------------------+
>
>      disp.BOXû0     ã turn off the display of boxes around data
>
>      disp.disp m    ã "display" sample data object in compact form
> 1 Lesha 10 Mandrake
> 2 Sasha 15 SuSE
> 3 Dima  5  RedHat
>      Òm
>  3 4
>
>      ã Writing the data to a file (from variable m):
>      `myfile à sys.exp m   ã write data to "myfile.m" dataset
>
>      ã (Next day):
>      ã Reading the data back from the file (into variable n):
>      nûsys.imp à `myfile   ã read data back in
>
>      disp.disp n
> 1 Lesha 10 Mandrake
> 2 Sasha 15 SuSE
> 3 Dima  5  RedHat
>      Òn
>  3 4
>
> I think the "sys.exp" and "sys.imp" functions are a good solution.
> But perhaps I don't completely understand what it is that you need
> to do.
>
> --Jon McGrew
>
>
>
> Alexei Zalivine wrote:
> >
> > Hello John,
> > >
> > > You can store a nested object by calling sys.exp{}, which
> encodes it into
> > > a character vector.  To convert it back into the nested object use
> > > sys.imp{}.
> > Oh, this way not very well. Because I want attach a variable and do
> > different actions with these data (search, replace, filtering
> and other).
> > After encode it into character vector I loose it. I will work with huge
> > amount of data (about handred Mbs per day), and I need the fast and flex
> > solution.
> > I will lookin for other way to store data.
> > >
> > > API's for external facilities like databases is usually done using
> > > dynamically loaded functions and the system function _dyld{}. See "C
> > > Subroutines" under Extensions and Tools in the manual.
> > Yes I read about _dyld system function, but I'm bad C programmer. May be
> > anybody wrote the cover functions for connect and executing SQL
> statement
> > for such DBs?
> >
> > Thank you,
> > Alexei.