/[cvs]/nfo/perl/libs/Data/Storage.pm
ViewVC logotype

Diff of /nfo/perl/libs/Data/Storage.pm

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.14 by joko, Thu Dec 19 16:27:59 2002 UTC revision 1.15 by joko, Sun Jan 19 03:12:59 2003 UTC
# Line 1  Line 1 
1  # $Id$  ## ------------------------------------------------------------------------
2  #  ##
3  # Copyright (c) 2002  Andreas Motl <andreas.motl@ilo.de>  ##    $Id$
4  #  ##
5  # See COPYRIGHT section in pod text below for usage and distribution rights.  ##    Copyright (c) 2002  Andreas Motl <andreas.motl@ilo.de>
6  #  ##
7  ############################################  ##    See COPYRIGHT section in pod text below for usage and distribution rights.
8  #  ##
9  #  $Log$  ## ------------------------------------------------------------------------
10  #  Revision 1.14  2002/12/19 16:27:59  joko  ##
11  #  - moved 'sub dropDb' to Data::Storage::Handler::DBI  ##  $Log$
12  #  ##  Revision 1.15  2003/01/19 03:12:59  joko
13  #  Revision 1.13  2002/12/17 21:54:12  joko  ##  + modified header
14  #  + feature when using Tangram:  ##  - removed pod-documentation - now in 'Storage.pod'
15  #    + what? each object created should delivered with a globally(!?) unique identifier (GUID) besides the native tangram object id (OID)  ##
16  #        + patched Tangram::Storage (jonen)  ##  Revision 1.14  2002/12/19 16:27:59  joko
17  #        + enhanced Data::Storage::Schema::Tangram (joko)  ##  - moved 'sub dropDb' to Data::Storage::Handler::DBI
18  #        + enhanced Data::Storage::Handler::Tangram 'sub getObjectByGuid' (jonen)  ##
19  #    + how?  ##  Revision 1.13  2002/12/17 21:54:12  joko
20  #        + each concrete (non-abstract) class gets injected with an additional field/property called 'guid' - this is done (dynamically) on schema level  ##  + feature when using Tangram:
21  #        + this property ('guid') gets filled on object creation/insertion from 'sub Tangram::Storage::_insert' using Data::UUID from CPAN  ##    + what? each object created should delivered with a globally(!?) unique identifier (GUID) besides the native tangram object id (OID)
22  #        + (as for now) this property can get accessed by calling 'getObjectByGuid' on the already known storage-handle used throughout the application  ##        + patched Tangram::Storage (jonen)
23  #  ##        + enhanced Data::Storage::Schema::Tangram (joko)
24  #  Revision 1.12  2002/12/12 02:50:15  joko  ##        + enhanced Data::Storage::Handler::Tangram 'sub getObjectByGuid' (jonen)
25  #  + this now (unfortunately) needs DBI for some helper functions  ##    + how?
26  #  + TODO: these have to be refactored to another scope! (soon!)  ##        + each concrete (non-abstract) class gets injected with an additional field/property called 'guid' - this is done (dynamically) on schema level
27  #  ##        + this property ('guid') gets filled on object creation/insertion from 'sub Tangram::Storage::_insert' using Data::UUID from CPAN
28  #  Revision 1.11  2002/12/11 06:53:19  joko  ##        + (as for now) this property can get accessed by calling 'getObjectByGuid' on the already known storage-handle used throughout the application
29  #  + updated pod  ##
30  #  ##  Revision 1.12  2002/12/12 02:50:15  joko
31  #  Revision 1.10  2002/12/07 03:37:23  joko  ##  + this now (unfortunately) needs DBI for some helper functions
32  #  + updated pod  ##  + TODO: these have to be refactored to another scope! (soon!)
33  #  ##
34  #  Revision 1.9  2002/12/01 22:15:45  joko  ##  Revision 1.11  2002/12/11 06:53:19  joko
35  #  - sub createDb: moved to handler  ##  + updated pod
36  #  ##
37  #  Revision 1.8  2002/11/29 04:48:23  joko  ##  Revision 1.10  2002/12/07 03:37:23  joko
38  #  + updated pod  ##  + updated pod
39  #  ##
40  #  Revision 1.7  2002/11/17 06:07:18  joko  ##  Revision 1.9  2002/12/01 22:15:45  joko
41  #  + creating the handler is easier than proposed first - for now :-)  ##  - sub createDb: moved to handler
42  #  + sub testAvailability  ##
43  #  ##  Revision 1.8  2002/11/29 04:48:23  joko
44  #  Revision 1.6  2002/11/09 01:04:58  joko  ##  + updated pod
45  #  + updated pod  ##
46  #  ##  Revision 1.7  2002/11/17 06:07:18  joko
47  #  Revision 1.5  2002/10/29 19:24:18  joko  ##  + creating the handler is easier than proposed first - for now :-)
48  #  - reduced logging  ##  + sub testAvailability
49  #  + added some pod  ##
50  #  ##  Revision 1.6  2002/11/09 01:04:58  joko
51  #  Revision 1.4  2002/10/27 18:35:07  joko  ##  + updated pod
52  #  + added pod  ##
53  #  ##  Revision 1.5  2002/10/29 19:24:18  joko
54  #  Revision 1.3  2002/10/25 11:40:37  joko  ##  - reduced logging
55  #  + enhanced robustness  ##  + added some pod
56  #  + more logging for debug-levels  ##
57  #  + sub dropDb  ##  Revision 1.4  2002/10/27 18:35:07  joko
58  #  ##  + added pod
59  #  Revision 1.2  2002/10/17 00:04:29  joko  ##
60  #  + sub createDb  ##  Revision 1.3  2002/10/25 11:40:37  joko
61  #  + sub isConnected  ##  + enhanced robustness
62  #  + bugfixes regarding "deep recursion" stuff  ##  + more logging for debug-levels
63  #  ##  + sub dropDb
64  #  Revision 1.1  2002/10/10 03:43:12  cvsjoko  ##
65  #  + new  ##  Revision 1.2  2002/10/17 00:04:29  joko
66  #  ##  + sub createDb
67  ############################################  ##  + sub isConnected
68    ##  + bugfixes regarding "deep recursion" stuff
69    ##
70    ##  Revision 1.1  2002/10/10 03:43:12  cvsjoko
71    ##  + new
72    ## ------------------------------------------------------------------------
73    
74    
75  BEGIN {  BEGIN {
76    $Data::Storage::VERSION = 0.02;    $Data::Storage::VERSION = 0.03;
77  }  }
78    
   
 =head1 NAME  
   
   Data::Storage - Interface for accessing various Storage implementations for Perl in an independent way  
   
   
 =head1 AIMS  
   
   - should encapsulate Tangram, DBI, DBD::CSV and LWP:: to access them in an unordinary (more convenient) way ;)  
   - introduce a generic layered structure, refactor *SUBLAYER*-stuff, make (e.g.) this possible:  
     Perl Data::Storage[DBD::CSV]  ->  Perl LWP::  ->  Internet HTTP/FTP/*  ->  Host Daemon  ->  csv-file  
   - provide generic synchronization mechanisms across arbitrary/multiple storages based on ident/checksum  
     maybe it's possible to have schema-, structural- and semantical modifications synchronized???  
   
   
 =head1 SYNOPSIS  
   
 =head2 BASIC ACCESS  
   
 =head2 ADVANCED ACCESS  
   
   ... via inheritance:  
     
     use Data::Storage;  
     my $proxyObj = new HttpProxy;  
     $proxyObj->{url} = $url;  
     $proxyObj->{payload} = $content;  
     $self->{storage}->insert($proxyObj);  
       
     use Data::Storage;  
     my $proxyObj = HttpProxy->new(  
       url => $url,  
       payload => $content,  
     );  
     $self->{storage}->insert($proxyObj);  
   
   
 =head2 SYNCHRONIZATION  
   
   my $nodemapping = {  
     'LangText' => 'langtexts.csv',  
     'Currency' => 'currencies.csv',  
     'Country'  => 'countries.csv',  
   };  
   
   my $propmapping = {  
     'LangText' => [  
       [ 'source:lcountrykey'  =>  'target:country' ],  
       [ 'source:lkey'         =>  'target:key' ],  
       [ 'source:lvalue'       =>  'target:text' ],  
     ],  
     'Currency' => [  
       [ 'source:ckey'         =>  'target:key' ],  
       [ 'source:cname'        =>  'target:text' ],  
     ],  
     'Country' => [  
       [ 'source:ckey'         =>  'target:key' ],  
       [ 'source:cname'        =>  'target:text' ],  
     ],  
   };  
   
   sub syncResource {  
   
     my $self = shift;  
     my $node_source = shift;  
     my $mode = shift;  
     my $opts = shift;  
       
     $mode ||= '';  
     $opts->{erase} ||= 0;  
       
     $logger->info( __PACKAGE__ . "->syncResource( node_source $node_source mode $mode erase $opts->{erase} )");  
     
     # resolve metadata for syncing requested resource  
     my $node_target = $nodemapping->{$node_source};  
     my $mapping = $propmapping->{$node_source};  
       
     if (!$node_target || !$mapping) {  
       # loggger.... "no target, sorry!"  
       print "error while resolving resource metadata", "\n";  
       return;  
     }  
       
     if ($opts->{erase}) {  
       $self->_erase_all($node_source);  
     }  
     
     # create new sync object  
     my $sync = Data::Transfer::Sync->new(  
       storages => {  
         L => $self->{bizWorks}->{backend},  
         R => $self->{bizWorks}->{resources},  
       },  
       id_authorities        =>  [qw( L ) ],  
       checksum_authorities  =>  [qw( L ) ],  
       write_protected       =>  [qw( R ) ],  
       verbose               =>  1,  
     );  
       
     # sync  
     # todo: filter!?  
     $sync->syncNodes( {  
       direction       =>  $mode,                 # | +PUSH | +PULL | -FULL | +IMPORT | -EXPORT  
       method          =>  'checksum',            # | -timestamp | -manual  
       source          =>  "L:$node_source",  
       source_ident    =>  'storage_method:id',  
       source_exclude  =>  [qw( id cs )],  
       target          =>  "R:$node_target",  
       target_ident    =>  'property:oid',  
       mapping         =>  $mapping,  
     } );  
   
   }  
   
   
 =head2 NOTE  
   
   This module heavily relies on DBI and Tangram, but adds a lot of additional bugs and quirks.  
   Please look at their documentation and/or this code for additional information.  
   
   
 =head1 REQUIREMENTS  
   
   For full functionality:  
     DBI              from CPAN  
     DBD::mysql       from CPAN  
     Tangram 2.04     from CPAN         (hmmm, 2.04 won't do in some cases)  
     Tangram 2.05     from http://...   (2.05 seems okay but there are also additional patches from our side)  
     Class::Tangram   from CPAN  
     DBD::CSV         from CPAN  
     MySQL::Diff      from http://adamspiers.org/computing/mysqldiff/  
     ... and all their dependencies  
   
 =cut  
   
 # The POD text continues at the end of the file.  
   
   
79  package Data::Storage;  package Data::Storage;
80    
81  use strict;  use strict;
# Line 234  sub new { Line 101  sub new {
101    my $arg_locator = shift;    my $arg_locator = shift;
102    my $arg_options = shift;    my $arg_options = shift;
103    
104      if (!$arg_locator) {
105        $logger->critical( __PACKAGE__ . "->new: No locator passed in!" );
106        return;
107      }
108    
109    #my $self = { STORAGEHANDLE => undef, @_ };    #my $self = { STORAGEHANDLE => undef, @_ };
110    my $self = { STORAGEHANDLE => undef, locator => $arg_locator, options => $arg_options };    my $self = { STORAGEHANDLE => undef, locator => $arg_locator, options => $arg_options };
111    #$logger->debug( __PACKAGE__ . "[$self->{locator}->{type}]" . "->new(@_)" );    #$logger->debug( __PACKAGE__ . "[$self->{locator}->{type}]" . "->new(@_)" );
# Line 279  sub AUTOLOAD { Line 151  sub AUTOLOAD {
151    if ($self->_filter_AUTOLOAD($method)) {    if ($self->_filter_AUTOLOAD($method)) {
152      #print "_accessStorage\n";      #print "_accessStorage\n";
153      $self->_accessStorage();      $self->_accessStorage();
154      $self->{STORAGEHANDLE}->$method(@_);      return $self->{STORAGEHANDLE}->$method(@_);
155    }    }
156        
157  }  }
# Line 417  sub testDsn { Line 289  sub testDsn {
289    
290  1;  1;
291  __END__  __END__
   
   
 =head1 DESCRIPTION  
   
 =head2 Data::Storage  
   
   Data::Storage is a module for accessing various "data structures / kinds of structured data" stored inside  
   various "data containers".  
   We tried to use the AdapterPattern (http://c2.com/cgi/wiki?AdapterPattern) to implement a wrapper-layer  
   around core CPAN modules (Tangram, DBI).  
   
 =head2 Why?  
   
   You will get a better code-structure (not bad for later maintenance) in growing Perl code projects,  
   especially when using multiple database connections at the same time.  
   You will be able to switch between different _kinds_ of implementations used for storing data.  
   Your code will use the very same API to access these storage layers.  
       ... implementation has to be changed for now  
   Maybe you will be able to switch "on-the-fly" without changing any bits in code in the future....  
       ... but that's not the focus  
   
 =head2 What else?  
   
   Having this, we were able to do implement a generic data synchronization module more easy,  
   please look at Data::Transfer.  
   
   
 =head1 AUTHORS / COPYRIGHT  
   
   The Data::Storage module is Copyright (c) 2002 Andreas Motl.  
   All rights reserved.  
   You may distribute it under the terms of either the GNU General Public  
   License or the Artistic License, as specified in the Perl README file.  
   
   
 =head1 ACKNOWLEDGEMENTS  
   
   Larry Wall for Perl, Tim Bunce for DBI, Jean-Louis Leroy for Tangram and Set::Object,  
   Sam Vilain for Class::Tangram, Jochen Wiedmann and Jeff Zucker for DBD::CSV & Co.,  
   Adam Spiers for MySQL::Diff and all contributors.  
   
   
 =head1 SUPPORT / WARRANTY  
   
   Data::Storage is free software. IT COMES WITHOUT WARRANTY OF ANY KIND.  
   
   
 =head1 TODO  
   
   
 =head2 BUGS  
   
 "DBI-Error [Tangram]: DBD::mysql::st execute failed: Unknown column 't1.requestdump' in 'field list'"  
   
   ... occours when operating on object-attributes not introduced yet:  
   this should be detected and appended/replaced through:  
   "Schema-Error detected, maybe (just) an inconsistency.  
   Please check if your declaration in schema-module "a" matches structure in database "b" or try to run"  
   db_setup.pl --dbkey=import --action=deploy  
   
   
 Compare schema (structure diff) with database ...  
   
   ... when issuing "db_setup.pl --dbkey=import --action=deploy"  
   on a database with an already deployed schema, use an additional "--update" then  
   to lift the schema inside the database to the current declared schema.  
   You will have to approve removals and changes on field-level while  
   new objects and new fields are introduced silently without any interaction needed.  
   In future versions there may be additional options to control silent processing of  
   removals and changes.  
   See this CRUD-table applying to the actions occouring on Classes and Class variables when deploying schemas,  
   don't mix this up with CRUD-actions on Objects, these are already handled by (e.g.) Tangram itself.  
   Classes:  
     C create    ->  yes, handled automatically  
     R retrieve  ->  no, not subject of this aspect since it is about deployment only  
     U update    ->  yes, automatically for Class meta-attributes, yes/no for Class variables (look at the rules down here)  
     D delete    ->  yes, just by user-interaction  
   Class variables:  
     C create    ->  yes, handled automatically  
     R retrieve  ->  no, not subject of this aspect since it is about deployment only  
     U update    ->  yes, just by user-interaction; maybe automatically if it can be determined that data wouldn't be lost  
     D delete    ->  yes, just by user-interaction  
     
   It's all about not to be able to loose data simply while this is in pre-alpha stage.  
   And loosing data by being able to modify and redeploy schemas easily is definitely quite easy.  
     
   As we can see, creations of Classes and new Class variables is handled  
   automatically and this is believed to be the most common case under normal circumstances.  
   
   
 =head2 FEATURES  
   
   - Get this stuff together with UML (Unified Modeling Language) and/or standards from ODMG.  
   - Make it possible to load/save schemas in XMI (XML Metadata Interchange),  
     which seems to be most commonly used today, perhaps handle objects with OIFML.  
     Integrate/bundle this with a web-/html-based UML modeling tool or  
     some other interesting stuff like the "Co-operative UML Editor" from Uni Darmstadt. (web-/java-based)  
   - Enable Round Trip Engineering. Keep code and diagrams in sync. Don't annoy/bother the programmers.  
   - Add support for some more handlers/locators to be able to  
      access the following standards/protocols/interfaces/programs/apis transparently:  
     +  DBD::CSV (via Data::Storage::Handler::DBI)  
    (-) Text::CSV, XML::CSV, XML::Excel  
     -  MAPI  
     -  LDAP  
     -  DAV (look at PerlDAV: http://www.webdav.org/perldav/)  
     -  Mbox (use formail for seperating/splitting entries/nodes)  
     -  Cyrus (cyrdeliver - what about cyrretrieve (export)???)  
     -  use File::DiffTree, use File::Compare  
     -  Hibernate  
     -  "Win32::UserAccountDb"  
     -  "*nix::UserAccountDb"  
     -  .wab - files (Windows Address Book)  
     -  .pst - files (Outlook Post Storage?)  
     -  XML (e.g. via XML::Simple?)  
   - Move to t3, look at InCASE  
   - some kind of security layer for methods/objects  
     - acls (stored via tangram/ldap?) for functions, methods and objects (entity- & data!?)  
     - where are the hooks needed then?  
       - is Data::Storage & Co. okay, or do we have to touch the innards of DBI and/or Tangram?  
       - an attempt to start could be:  
          - 'sub getACLByObjectId($id, $context)'  
          - 'sub getACLByMethodname($id, $context)'  
          - 'sub getACLByName($id, $context)'  
             ( would require a kinda registry to look up these very names pointing to arbitrary locations (code, data, ...) )  
   - add more hooks and various levels  
   - better integrate introduced 'getObjectByGuid'-mechanism from Data::Storage::Handler::Tangram  
   
   
 =head3 LINKS / REFERENCES  
   
   Specs:  
     UML 1.3 Spec: http://cgi.omg.org/cgi-bin/doc?ad/99-06-08.pdf  
     XMI 1.1 Spec: http://cgi.omg.org/cgi-bin/doc?ad/99-10-02.pdf  
     XMI 2.0 Spec: http://cgi.omg.org/docs/ad/01-06-12.pdf  
     ODMG: http://odmg.org/  
     OIFML: http://odmg.org/library/readingroom/oifml.pdf  
   
   CASE Tools:  
     Rational Rose (commercial): http://www.rational.com/products/rose/  
     Together (commercial): http://www.oi.com/products/controlcenter/index.jsp  
     InCASE - Tangram-based Universal Object Editor  
     Sybase PowerDesigner: http://www.sybase.com/powerdesigner  
     
   UML Editors:  
     Fujaba (free, university): http://www.fujaba.de/  
     ArgoUML (free): http://argouml.tigris.org/  
     Poseidon (commercial): http://www.gentleware.com/products/poseidonDE.php3  
     Co-operative UML Editor (research): http://www.darmstadt.gmd.de/concert/activities/internal/umledit.html  
     Metamill (commercial): http://www.metamill.com/  
     Violet (university, research, education): http://www.horstmann.com/violet/  
     PyUt (free): http://pyut.sourceforge.net/  
     (Dia (free): http://www.lysator.liu.se/~alla/dia/)  
     UMLet (free, university): http://www.swt.tuwien.ac.at/umlet/index.html  
     Voodoo (free): http://voodoo.sourceforge.net/  
     Umbrello UML Modeller: http://uml.sourceforge.net/  
   
   UML Tools:  
     http://www.objectsbydesign.com/tools/umltools_byPrice.html  
   
   Further readings:  
     http://www.google.com/search?q=web+based+uml+editor&hl=en&lr=&ie=UTF-8&oe=UTF-8&start=10&sa=N  
     http://www.fernuni-hagen.de/DVT/Aktuelles/01FHHeidelberg.pdf  
     http://www.enhyper.com/src/documentation/  
     http://cis.cs.tu-berlin.de/Dokumente/Diplomarbeiten/2001/skinner.pdf  
     http://citeseer.nj.nec.com/vilain00diagrammatic.html  
     http://archive.devx.com/uml/articles/Smith01/Smith01-3.asp  
   
Legend:
Removed from v.1.14  
changed lines
  Added in v.1.15
MailToCvsAdmin">MailToCvsAdmin
 ViewVC Help
Powered by ViewVC 1.1.26 RSS 2.0 feed