--- nfo/perl/libs/Data/Transfer/Sync.README.text 2003/01/19 01:23:03 1.2 +++ nfo/perl/libs/Data/Transfer/Sync.README.text 2003/02/09 16:47:41 1.5 @@ -1,28 +1,103 @@ +NEWS + 2003-02-09 - backward compatibility with custom application + + - Tangram <-> .csv + - Tangram <-> mysql + - Tangram <-> perl-code + + 2003-01-18 - start integration with Torus + + - MAPI <-> LDAP + - please visit http://www.netfrag.org/horde/chora/cvs.php/perl/scripts/outlook2ldap/ + + 2002-10-10 - initial commit + + - Tangram <-> mysql + DESCRIPTION - Data::Transfer::Sync is a module providing a generic synchronization - process across arbitrary/multiple storages based on a ident/checksum - mechanism. It sits on top of Data::Storage. + Data::Transfer::Sync is a module providing a generic synchronization process + across arbitrary/multiple storages based on a ident/checksum mechanism. + It sits on top of Data::Storage. REQUIREMENTS - For full functionality: + dependencies + + from CPAN or (for win32) a ppm-repository: + App::Config (CPAN only) + Tie::IxHash + Log::Dispatch + Log::Dispatch::Config + mixin + Hash::Merge + Tie::SecureHash + Date::Manip + + from nfo/perl/libs: + Data::Compare Data::Storage Data::Transform - Data::Compare - ... and all their dependencies + DesignPattern:: (to be renamed into Class::) + ... and all their dependencies. -AUTHORS / COPYRIGHT - The Data::Storage module is Copyright (c) 2002 Andreas Motl. All rights - reserved. + additional modules required for 'outlook2ldap' - You may distribute it under the terms of either the GNU General Public - License or the Artistic License, as specified in the Perl README file. + Win32::OLE + Net::LDAP + + additional modules required for 'db2db' + + DBI + DBD::mysql + DBD::CSV + Tangram + +AUTHORS / COPYRIGHT / LICENSE + The Data::Transfer::Sync module is Copyright (c) 2002 Andreas Motl. + All rights reserved. + + Code contributions, suggestions, support and ideas by + Sebastian Utz and Jan Hoffmann . + + You may distribute it under the terms of either the GNU General Public + License or the Artistic License, as specified in the Perl README file. SUPPORT / WARRANTY - Data::Storage is free software. IT COMES WITHOUT WARRANTY OF ANY KIND. + Data::Transfer::Sync is free software. IT COMES WITHOUT WARRANTY OF ANY KIND. + + Please feel free to contact me at . + +EXAMPLE OUTPUT CHUNKS + Synchronizing orm objects against entries in a .csv file - 17 items modified + + notice: PULL INTO backend NODE * TYPE Country SELECT NODE * TYPE [n/a] FROM base USING MODULE BizWorks::ResourceMapping; + notice: OEF::Script::AbstractFeeder: + - Loading data of type [n/a] and + filtered by * from the storage named base + to the storage named backend - filtered by *. + - Will attempt to convert the data to Country. + info: Data::Transfer::Sync::API->syncNodes: source=backend/* [Country] <- target=base/* [] + :c:c:c:c:c:c:c:ctr:c:c:c:c:c:c:c:ctr:c:c:c:c:c:c:ctr:c:ctr:c:c:c:c:c:c:c:c:c:c:c:ctr:c:c:c:ctr:c:c:c:c:c:c:c:ctr:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c + :c:c:c:c:c:c:c:c:ctr:c:c:c:c:c:c:c:c:c:c:c:ctr:c:c:c:c:ctr:c:c:c:ctr:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:ctr:ctr:c:c:c:c:ctr:c:c:c:c:c:c:c + :c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:ctr:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:ctr:c:c:ctr:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c + info: Data::Transfer::Sync::Core->_startsync: statistics: in_sync: 180 / total: 197 / attempt_modify: 17 / ok: 17 + + Synchronizing orm objects against entries in a .csv file - in-sync + + notice: PULL INTO backend NODE * TYPE Country SELECT NODE * TYPE [n/a] FROM base USING MODULE BizWorks::ResourceMapping; + notice: OEF::Script::AbstractFeeder: + - Loading data of type [n/a] and + filtered by * from the storage named base + to the storage named backend - filtered by *. + - Will attempt to convert the data to Country. + info: Data::Transfer::Sync::API->syncNodes: source=backend/* [Country] <- target=base/* [] + :c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c + :c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c + :c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c + info: Data::Transfer::Sync::Core->_startsync: statistics: in_sync: 197 / total: 197 BUGS When in "import" mode for windows file - DBD::AutoCSV may hang. - Hint: Maybe the source node contains an ident-, but no checksum column? + Hint: Maybe the source node contains an ident-, but no checksum-column? USER LEVEL ERRORS Mapping @@ -94,34 +169,23 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - TODO - - sub _resolveIdentProvider - - wrap _doModifySource and _doTransferTarget around a core function which can change virtually any type of node - - split this module up into Sync.pm, Sync/Core.pm, Sync/Compare.pm and Sync/Compare/Checksum.pm - - introduce _compareNodes as a core method and wrap it around methods in Sync/Compare/Checksum.pm - - introduce Sync/Compare/MyComparisonImplementation.pm - - some generic deferring method - e.g. "$self->defer(action)" - to be able to accumulate a bunch of actions for later processing + o sub _resolveIdentProvider + x wrap _doModifySource and _doTransferTarget around a core function which can change virtually any type of node + x split this module up into Sync.pm, Sync/Core.pm, Sync/Compare.pm and Sync/Compare/Checksum.pm + o introduce _compareNodes as a core method and wrap it around methods in Sync/Compare/Checksum.pm + x introduce Sync/Compare/MyComparisonImplementation.pm + o some generic deferring method - e.g. "$self->defer(action)" - to be able to accumulate a bunch of actions for later processing - this implies everything done is _really_ split up into generic actions - how else would we defer them??? - example uses: - fetch whole checksum list from node - remember source ident retransmits - remember: this is convenient - and maybe / of course faster - but we'll loose "per-node-atomic" operations - - feature: mechanism to implicit inject checksum property to nodes (alter table / modify schema) - - expand statistics / keep track of: + o feature: mechanism to implicit inject checksum property to nodes (alter table / modify schema) + o expand statistics / keep track of: - touched/untouched nodes - - full sync + o full sync - just do a push and a pull for now but use stats for touched nodes in between to speed up things - - introduce some new metadata flags for a synchronization partner which is (e.g.) of "source" or "target": + o introduce some new metadata flags for a descent - isNewNodePropagator - - isWriteProtected - - add additional mechanisms: "how is the sync done exactly?" - + based on ident/checksum (this compares uniqueness) - - based on ident/timestamp (this compares up-to-date-ness) - - based on abstract rules (free, script, ..., maybe taken from a rulebase) ... - ... combining queries on metadata (e.g. flags) associated with the object (may also be stored inside the object itself for convenience) - => (example): - $admin = new Person( admin => 1); - $user1 = new Person(); - $user2 = new Person( watched_by => $admin ); - $context = new Context( postmaster => $admin ); - $data = new Data( created_by => $user2, crud_hooks => { u => $admin } ); - $admin-> \ No newline at end of file + x isWriteProtected +