/[cvs]/nfo/perl/libs/Data/Transfer/Sync.pod
ViewVC logotype

Diff of /nfo/perl/libs/Data/Transfer/Sync.pod

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.2 by joko, Sun Feb 9 04:56:22 2003 UTC revision 1.6 by joko, Sat Feb 22 18:25:38 2003 UTC
# Line 4  Line 4 
4  =head1 NEWS  =head1 NEWS
5    
6    
7    =head4 2003-02-22  -  enabled multiple task in-process synchronization ...
8    
9      x using Data::Rap and rap.pl
10      x broke sync.pl...
11      o ... re-enable via Data::Transfer::Sync::API::V1?
12    
13    
14  =head4 2003-02-09  -  backward compatibility with custom application  =head4 2003-02-09  -  backward compatibility with custom application
15    
16    - Tangram <-> .csv    - Tangram <-> .csv
# Line 11  Line 18 
18    - Tangram <-> perl-code    - Tangram <-> perl-code
19    
20    
21  =head4 2003-01-18  -  start integration with Torus  =head4 2003-01-18  -  started integration with Torus
22    
23    - MAPI <-> LDAP    - MAPI <-> LDAP
24    - please look at http://www.netfrag.org/horde/chora/cvs.php/perl/scripts/outlook2ldap/    - please visit http://www.netfrag.org/horde/chora/cvs.php/perl/scripts/outlook2ldap/
25    
26    
27  =head4 2002-10-10  -  initial commit  =head4 2002-10-10  -  initial commit
# Line 33  Line 40 
40    
41  =head1 REQUIREMENTS  =head1 REQUIREMENTS
42    
43    For full functionality:  =head3 dependencies
44    
45      from CPAN or (for win32) a ppm-repository:
46        App::Config (CPAN only)
47        Tie::IxHash
48        Log::Dispatch
49        Log::Dispatch::Config
50        mixin
51        Hash::Merge
52        Tie::SecureHash
53        Date::Manip
54        Digest::MD5
55        ... and maybe some more.
56    
57      from nfo/perl/libs:
58      Data::Compare      Data::Compare
59      Data::Storage      Data::Storage
60      Data::Transform      Data::Transform
61      ... and all their dependencies.      DesignPattern:: (to be renamed into Class::)
62        ... and all their dependencies. (like Data::UUID & Co.)
63    
64    
65    =head3 additional modules required for 'outlook2ldap'
66    
67        Win32::OLE
68        Net::LDAP
69    
70    =head3 additional modules required for 'db2db'
71    
72        DBI
73        DBD::mysql
74        DBD::CSV
75        Tangram
76    
77    
78    
79  =head1 AUTHORS / COPYRIGHT / LICENSE  =head1 AUTHORS / COPYRIGHT / LICENSE
80    
81    The Data::Transfer::Sync module is Copyright (c) 2002 Andreas Motl.    The Data::Transfer::Sync module is Copyright (c) 2002, 2003 Andreas Motl.
82    All rights reserved.    All Rights Reserved.
83    
84    Code contributions, suggestions, support and ideas by    Code contributions, suggestions, support and ideas by
85    Sebastian Utz <su@tunemedia.de> and Jan Hoffmann <jan@romain-volk.com>.    Sebastian Utz <su@tunemedia.de> and Jan Hoffmann <janosch@ultrajan.de>.
86        
87    You may distribute it under the terms of either the GNU General Public    You may distribute it under the terms of either the GNU General Public
88    License or the Artistic License, as specified in the Perl README file.    License or the Artistic License, as specified in the Perl README file.
89    
90    
91    =head1 COPYRIGHT NOTE
92    
93      * CAESAR - Collaborative And Easy System ARchitecture
94      *
95      * Data Synchronization Module
96      *
97      * Permission to use, copy, modify and distribute this software and its
98      * documentation is hereby granted, provided that both the copyright
99      * notice and this permission notice appear in all copies of the
100      * software, derivative works or modified versions, and any portions
101      * thereof, and that both notices appear in supporting documentation.
102      *
103      * NETFRAG.ORG AND THE AUTHOR ALLOW FREE USE OF THIS SOFTWARE
104      * IN ITS "AS IS" CONDITION. WE DISCLAIM ANY LIABILITY OF ANY KIND
105      * FOR ANY DAMAGES WHATSOEVER RESULTING FROM THE USE OF THIS SOFTWARE.
106    
107    
108    
109  =head1 SUPPORT / WARRANTY  =head1 SUPPORT / WARRANTY
110    
111    Data::Transfer::Sync is free software. IT COMES WITHOUT WARRANTY OF ANY KIND.    Data::Transfer::Sync is free software. IT COMES WITHOUT WARRANTY OF ANY KIND.
112    
113    Please feel free to contact me at <andreas.motl@ilo.de>.    Please feel free to contact me at <andreas.motl@ilo.de>.
114    
115      Any improvements and contributions are greatly appreciated.
116    
117      We have set up a newsgroup. (currently not much traffic)
118      Be welcome to visit news://news.netfrag.org/nfo.caesar or - if your system
119      lacks a newsreader - try http://news.netfrag.org/nfo.caesar .
120      
121    
122    
123    
124    =head1 TODO
125    
126    =head4 overall
127    
128      o try to get this stuff together with SyncML (http://syncml.org/) somehow ...?
129    
130    
131    =head4 this code
132    
133      o sub _resolveIdentProvider
134      x wrap _doModifySource and _doTransferTarget around a core function which can change virtually any type of node
135      x split this module up into Sync.pm, Sync/Core.pm, Sync/Compare.pm and Sync/Compare/Checksum.pm
136      o introduce _compareNodes as a core method and wrap it around methods in Sync/Compare/Checksum.pm
137      x introduce Sync/Compare/MyComparisonImplementation.pm
138      o some generic deferring method - e.g. "$self->defer(action)" - to be able to accumulate a bunch of actions for later processing
139         - this implies everything done is _really_ split up into generic actions - how else would we defer them???
140         - example uses:
141            - fetch whole checksum list from node
142            - remember source ident retransmits
143         - remember: this is convenient - and maybe / of course faster - but we'll loose "per-node-atomic" operations
144      o feature: mechanism to implicit inject checksum property to nodes (alter table / modify schema)
145      o expand statistics / keep track of:
146        - touched/untouched nodes
147      o full sync
148        - just do a push and a pull for now but use stats for touched nodes in between to speed up things
149      o introduce some new metadata flags for a descent
150        - isNewNodePropagator
151        x isWriteProtected
152    
153    
154    
155  =head1 EXAMPLE OUTPUT CHUNKS  =head1 EXAMPLE OUTPUT CHUNKS
156    
157  =head2 Synchronizing orm objects against entries in a .csv file - 17 items modified  =head2 Synchronizing orm objects against entries in a .csv file - 17 items modified
158    
159    notice: PULL INTO backend NODE * TYPE Country SELECT NODE * TYPE [n/a] FROM base USING MODULE BizWorks::ResourceMapping;    notice: PULL INTO backend NODE * TYPE Country SELECT NODE * TYPE [n/a] FROM base USING MODULE MyApp::ResourceMapping;
160    notice: OEF::Script::AbstractFeeder:    notice: OEF::Script::AbstractFeeder:
161        - Loading data of type [n/a] and        - Loading data of type [n/a] and
162          filtered by * from the storage named base          filtered by * from the storage named base
# Line 76  Line 168 
168    :c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:ctr:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:ctr:c:c:ctr:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c    :c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:ctr:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:ctr:c:c:ctr:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c
169    info: Data::Transfer::Sync::Core->_startsync: statistics: in_sync: 180  /  total: 197  /  attempt_modify: 17  /  ok: 17    info: Data::Transfer::Sync::Core->_startsync: statistics: in_sync: 180  /  total: 197  /  attempt_modify: 17  /  ok: 17
170    
171  =head2 Synchronizing orm objects against entries in a .csv file - in-sync  =head2 Synchronizing orm objects against entries in a .csv file - all items in-sync
172        
173    notice: PULL INTO backend NODE * TYPE Country SELECT NODE * TYPE [n/a] FROM base USING MODULE BizWorks::ResourceMapping;    notice: PULL INTO backend NODE * TYPE Country SELECT NODE * TYPE [n/a] FROM base USING MODULE MyApp::ResourceMapping;
174    notice: OEF::Script::AbstractFeeder:    notice: OEF::Script::AbstractFeeder:
175        - Loading data of type [n/a] and        - Loading data of type [n/a] and
176          filtered by * from the storage named base          filtered by * from the storage named base
# Line 103  Line 195 
195  =head4 Mapping  =head4 Mapping
196    
197    - - - - - - - - - - - - - - - - - - - - - - - - - -    - - - - - - - - - - - - - - - - - - - - - - - - - -
198    info: BizWorks::Process::Setup->syncResource( source_node Currency mode PULL erase 0 import 0 )critical: BizWorks::Process::Setup->startSync: Can't access mapping for node "Currency" - please check BizWorks::ResourceMapping.    info: MyApp::Process::Setup->syncResource( source_node Currency mode PULL erase 0 import 0 )critical: MyApp::Process::Setup->startSync: Can't access mapping for node "Currency" - please check MyApp::ResourceMapping.
199    - - - - - - - - - - - - - - - - - - - - - - - - - -    - - - - - - - - - - - - - - - - - - - - - - - - - -
200    You have to create a sub for each node used in synchronization inside named Perl module. The name of this sub _must_ match    You have to create a sub for each node used in synchronization inside named Perl module. The name of this sub _must_ match
201    the name of the node you want to sync. This sub holds mapping metadata to give the engine hints about how    the name of the node you want to sync. This sub holds mapping metadata to give the engine hints about how
# Line 114  Line 206 
206  =head4 DBD::AutoCSV's rulebase  =head4 DBD::AutoCSV's rulebase
207        
208    - - - - - - - - - - - - - - - - - - - - - - - - - -    - - - - - - - - - - - - - - - - - - - - - - - - - -
209    info: BizWorks::Process::Setup->syncResource( source_node Currency mode PULL erase 0 import 0 )    info: MyApp::Process::Setup->syncResource( source_node Currency mode PULL erase 0 import 0 )
210    info: Data::Transfer::Sync->syncNodes: source=L/Currency <- target=R/currencies.csv    info: Data::Transfer::Sync->syncNodes: source=L/Currency <- target=R/currencies.csv
211        
212    Execution ERROR: Error while scanning: Missing first row or scanrule not applied. at C:/home/amo/develop/netfrag.org/nfo/perl/libs/DBD/CSV.p    Execution ERROR: Error while scanning: Missing first row or scanrule not applied. at C:/home/amo/develop/netfrag.org/nfo/perl/libs/DBD/CSV.p
# Line 173  Line 265 
265    - - - - - - - - - - - - - - - - - - - - - - - - - -    - - - - - - - - - - - - - - - - - - - - - - - - - -
266    
267    
 =head1 TODO  
   
   o sub _resolveIdentProvider  
   x wrap _doModifySource and _doTransferTarget around a core function which can change virtually any type of node  
   x split this module up into Sync.pm, Sync/Core.pm, Sync/Compare.pm and Sync/Compare/Checksum.pm  
   o introduce _compareNodes as a core method and wrap it around methods in Sync/Compare/Checksum.pm  
   x introduce Sync/Compare/MyComparisonImplementation.pm  
   o some generic deferring method - e.g. "$self->defer(action)" - to be able to accumulate a bunch of actions for later processing  
      - this implies everything done is _really_ split up into generic actions - how else would we defer them???  
      - example uses:  
         - fetch whole checksum list from node  
         - remember source ident retransmits  
      - remember: this is convenient - and maybe / of course faster - but we'll loose "per-node-atomic" operations  
   o feature: mechanism to implicit inject checksum property to nodes (alter table / modify schema)  
   o expand statistics / keep track of:  
     - touched/untouched nodes  
   o full sync  
     - just do a push and a pull for now but use stats for touched nodes in between to speed up things  
   o introduce some new metadata flags for a descent  
     - isNewNodePropagator  
     x isWriteProtected  
268    
269    
270  =cut  =cut

Legend:
Removed from v.1.2  
changed lines
  Added in v.1.6

MailToCvsAdmin">MailToCvsAdmin
ViewVC Help
Powered by ViewVC 1.1.26 RSS 2.0 feed