/[cvs]/nfo/perl/libs/Data/Transfer/Sync.pod
ViewVC logotype

Diff of /nfo/perl/libs/Data/Transfer/Sync.pod

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.2 by joko, Sun Feb 9 04:56:22 2003 UTC revision 1.5 by joko, Tue Feb 11 10:28:03 2003 UTC
# Line 14  Line 14 
14  =head4 2003-01-18  -  start integration with Torus  =head4 2003-01-18  -  start integration with Torus
15    
16    - MAPI <-> LDAP    - MAPI <-> LDAP
17    - please look at http://www.netfrag.org/horde/chora/cvs.php/perl/scripts/outlook2ldap/    - please visit http://www.netfrag.org/horde/chora/cvs.php/perl/scripts/outlook2ldap/
18    
19    
20  =head4 2002-10-10  -  initial commit  =head4 2002-10-10  -  initial commit
# Line 33  Line 33 
33    
34  =head1 REQUIREMENTS  =head1 REQUIREMENTS
35    
36    For full functionality:  =head3 dependencies
37    
38      from CPAN or (for win32) a ppm-repository:
39        App::Config (CPAN only)
40        Tie::IxHash
41        Log::Dispatch
42        Log::Dispatch::Config
43        mixin
44        Hash::Merge
45        Tie::SecureHash
46        Date::Manip
47        Digest::MD5
48        ... and maybe some more.
49    
50      from nfo/perl/libs:
51      Data::Compare      Data::Compare
52      Data::Storage      Data::Storage
53      Data::Transform      Data::Transform
54      ... and all their dependencies.      DesignPattern:: (to be renamed into Class::)
55        ... and all their dependencies. (like Data::UUID & Co.)
56    
57    
58    =head3 additional modules required for 'outlook2ldap'
59    
60        Win32::OLE
61        Net::LDAP
62    
63    =head3 additional modules required for 'db2db'
64    
65        DBI
66        DBD::mysql
67        DBD::CSV
68        Tangram
69    
70    
71    
72  =head1 AUTHORS / COPYRIGHT / LICENSE  =head1 AUTHORS / COPYRIGHT / LICENSE
73    
74    The Data::Transfer::Sync module is Copyright (c) 2002 Andreas Motl.    The Data::Transfer::Sync module is Copyright (c) 2002, 2003 Andreas Motl.
75    All rights reserved.    All Rights Reserved.
76    
77    Code contributions, suggestions, support and ideas by    Code contributions, suggestions, support and ideas by
78    Sebastian Utz <su@tunemedia.de> and Jan Hoffmann <jan@romain-volk.com>.    Sebastian Utz <su@tunemedia.de> and Jan Hoffmann <janosch@ultrajan.de>.
79        
80    You may distribute it under the terms of either the GNU General Public    You may distribute it under the terms of either the GNU General Public
81    License or the Artistic License, as specified in the Perl README file.    License or the Artistic License, as specified in the Perl README file.
82    
83    
84    =head1 COPYRIGHT NOTE
85    
86      * CAESAR - Collaborative And Easy System ARchitecture
87      *
88      * Data Synchronization Module
89      *
90      * Permission to use, copy, modify and distribute this software and its
91      * documentation is hereby granted, provided that both the copyright
92      * notice and this permission notice appear in all copies of the
93      * software, derivative works or modified versions, and any portions
94      * thereof, and that both notices appear in supporting documentation.
95      *
96      * NETFRAG.ORG AND THE AUTHOR ALLOW FREE USE OF THIS SOFTWARE
97      * IN ITS "AS IS" CONDITION. WE DISCLAIM ANY LIABILITY OF ANY KIND
98      * FOR ANY DAMAGES WHATSOEVER RESULTING FROM THE USE OF THIS SOFTWARE.
99    
100    
101    
102  =head1 SUPPORT / WARRANTY  =head1 SUPPORT / WARRANTY
103    
104    Data::Transfer::Sync is free software. IT COMES WITHOUT WARRANTY OF ANY KIND.    Data::Transfer::Sync is free software. IT COMES WITHOUT WARRANTY OF ANY KIND.
105    
106    Please feel free to contact me at <andreas.motl@ilo.de>.    Please feel free to contact me at <andreas.motl@ilo.de>.
107    
108      Any improvements and contributions are greatly appreciated.
109    
110      We have set up a newsgroup. (currently not much traffic)
111      Be welcome to visit news://news.netfrag.org/nfo.caesar or - if your system
112      lacks a newsreader - try http://news.netfrag.org/nfo.caesar .
113      
114    
115    
116    
117    =head1 TODO
118    
119    =head4 overall
120    
121      o try to get this stuff together with SyncML (http://syncml.org/) somehow ...?
122    
123    
124    =head4 this code
125    
126      o sub _resolveIdentProvider
127      x wrap _doModifySource and _doTransferTarget around a core function which can change virtually any type of node
128      x split this module up into Sync.pm, Sync/Core.pm, Sync/Compare.pm and Sync/Compare/Checksum.pm
129      o introduce _compareNodes as a core method and wrap it around methods in Sync/Compare/Checksum.pm
130      x introduce Sync/Compare/MyComparisonImplementation.pm
131      o some generic deferring method - e.g. "$self->defer(action)" - to be able to accumulate a bunch of actions for later processing
132         - this implies everything done is _really_ split up into generic actions - how else would we defer them???
133         - example uses:
134            - fetch whole checksum list from node
135            - remember source ident retransmits
136         - remember: this is convenient - and maybe / of course faster - but we'll loose "per-node-atomic" operations
137      o feature: mechanism to implicit inject checksum property to nodes (alter table / modify schema)
138      o expand statistics / keep track of:
139        - touched/untouched nodes
140      o full sync
141        - just do a push and a pull for now but use stats for touched nodes in between to speed up things
142      o introduce some new metadata flags for a descent
143        - isNewNodePropagator
144        x isWriteProtected
145    
146    
147    
148  =head1 EXAMPLE OUTPUT CHUNKS  =head1 EXAMPLE OUTPUT CHUNKS
149    
150  =head2 Synchronizing orm objects against entries in a .csv file - 17 items modified  =head2 Synchronizing orm objects against entries in a .csv file - 17 items modified
151    
152    notice: PULL INTO backend NODE * TYPE Country SELECT NODE * TYPE [n/a] FROM base USING MODULE BizWorks::ResourceMapping;    notice: PULL INTO backend NODE * TYPE Country SELECT NODE * TYPE [n/a] FROM base USING MODULE MyApp::ResourceMapping;
153    notice: OEF::Script::AbstractFeeder:    notice: OEF::Script::AbstractFeeder:
154        - Loading data of type [n/a] and        - Loading data of type [n/a] and
155          filtered by * from the storage named base          filtered by * from the storage named base
# Line 76  Line 161 
161    :c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:ctr:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:ctr:c:c:ctr:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c    :c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:ctr:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:ctr:c:c:ctr:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c
162    info: Data::Transfer::Sync::Core->_startsync: statistics: in_sync: 180  /  total: 197  /  attempt_modify: 17  /  ok: 17    info: Data::Transfer::Sync::Core->_startsync: statistics: in_sync: 180  /  total: 197  /  attempt_modify: 17  /  ok: 17
163    
164  =head2 Synchronizing orm objects against entries in a .csv file - in-sync  =head2 Synchronizing orm objects against entries in a .csv file - all items in-sync
165        
166    notice: PULL INTO backend NODE * TYPE Country SELECT NODE * TYPE [n/a] FROM base USING MODULE BizWorks::ResourceMapping;    notice: PULL INTO backend NODE * TYPE Country SELECT NODE * TYPE [n/a] FROM base USING MODULE MyApp::ResourceMapping;
167    notice: OEF::Script::AbstractFeeder:    notice: OEF::Script::AbstractFeeder:
168        - Loading data of type [n/a] and        - Loading data of type [n/a] and
169          filtered by * from the storage named base          filtered by * from the storage named base
# Line 103  Line 188 
188  =head4 Mapping  =head4 Mapping
189    
190    - - - - - - - - - - - - - - - - - - - - - - - - - -    - - - - - - - - - - - - - - - - - - - - - - - - - -
191    info: BizWorks::Process::Setup->syncResource( source_node Currency mode PULL erase 0 import 0 )critical: BizWorks::Process::Setup->startSync: Can't access mapping for node "Currency" - please check BizWorks::ResourceMapping.    info: MyApp::Process::Setup->syncResource( source_node Currency mode PULL erase 0 import 0 )critical: MyApp::Process::Setup->startSync: Can't access mapping for node "Currency" - please check MyApp::ResourceMapping.
192    - - - - - - - - - - - - - - - - - - - - - - - - - -    - - - - - - - - - - - - - - - - - - - - - - - - - -
193    You have to create a sub for each node used in synchronization inside named Perl module. The name of this sub _must_ match    You have to create a sub for each node used in synchronization inside named Perl module. The name of this sub _must_ match
194    the name of the node you want to sync. This sub holds mapping metadata to give the engine hints about how    the name of the node you want to sync. This sub holds mapping metadata to give the engine hints about how
# Line 114  Line 199 
199  =head4 DBD::AutoCSV's rulebase  =head4 DBD::AutoCSV's rulebase
200        
201    - - - - - - - - - - - - - - - - - - - - - - - - - -    - - - - - - - - - - - - - - - - - - - - - - - - - -
202    info: BizWorks::Process::Setup->syncResource( source_node Currency mode PULL erase 0 import 0 )    info: MyApp::Process::Setup->syncResource( source_node Currency mode PULL erase 0 import 0 )
203    info: Data::Transfer::Sync->syncNodes: source=L/Currency <- target=R/currencies.csv    info: Data::Transfer::Sync->syncNodes: source=L/Currency <- target=R/currencies.csv
204        
205    Execution ERROR: Error while scanning: Missing first row or scanrule not applied. at C:/home/amo/develop/netfrag.org/nfo/perl/libs/DBD/CSV.p    Execution ERROR: Error while scanning: Missing first row or scanrule not applied. at C:/home/amo/develop/netfrag.org/nfo/perl/libs/DBD/CSV.p
# Line 173  Line 258 
258    - - - - - - - - - - - - - - - - - - - - - - - - - -    - - - - - - - - - - - - - - - - - - - - - - - - - -
259    
260    
 =head1 TODO  
   
   o sub _resolveIdentProvider  
   x wrap _doModifySource and _doTransferTarget around a core function which can change virtually any type of node  
   x split this module up into Sync.pm, Sync/Core.pm, Sync/Compare.pm and Sync/Compare/Checksum.pm  
   o introduce _compareNodes as a core method and wrap it around methods in Sync/Compare/Checksum.pm  
   x introduce Sync/Compare/MyComparisonImplementation.pm  
   o some generic deferring method - e.g. "$self->defer(action)" - to be able to accumulate a bunch of actions for later processing  
      - this implies everything done is _really_ split up into generic actions - how else would we defer them???  
      - example uses:  
         - fetch whole checksum list from node  
         - remember source ident retransmits  
      - remember: this is convenient - and maybe / of course faster - but we'll loose "per-node-atomic" operations  
   o feature: mechanism to implicit inject checksum property to nodes (alter table / modify schema)  
   o expand statistics / keep track of:  
     - touched/untouched nodes  
   o full sync  
     - just do a push and a pull for now but use stats for touched nodes in between to speed up things  
   o introduce some new metadata flags for a descent  
     - isNewNodePropagator  
     x isWriteProtected  
261    
262    
263  =cut  =cut

Legend:
Removed from v.1.2  
changed lines
  Added in v.1.5

MailToCvsAdmin">MailToCvsAdmin
ViewVC Help
Powered by ViewVC 1.1.26 RSS 2.0 feed