1 |
|
NEWS |
2 |
|
2003-02-09 - backward compatibility with custom application |
3 |
|
|
4 |
|
- Tangram <-> .csv |
5 |
|
- Tangram <-> mysql |
6 |
|
- Tangram <-> perl-code |
7 |
|
|
8 |
|
2003-01-18 - start integration with Torus |
9 |
|
|
10 |
|
- MAPI <-> LDAP |
11 |
|
- please visit http://www.netfrag.org/horde/chora/cvs.php/perl/scripts/outlook2ldap/ |
12 |
|
|
13 |
|
2002-10-10 - initial commit |
14 |
|
|
15 |
|
- Tangram <-> mysql |
16 |
|
|
17 |
DESCRIPTION |
DESCRIPTION |
18 |
Data::Transfer::Sync is a module providing a generic synchronization |
Data::Transfer::Sync is a module providing a generic synchronization process |
19 |
process across arbitrary/multiple storages based on a ident/checksum |
across arbitrary/multiple storages based on a ident/checksum mechanism. |
20 |
mechanism. It sits on top of Data::Storage. |
It sits on top of Data::Storage. |
21 |
|
|
22 |
REQUIREMENTS |
REQUIREMENTS |
23 |
For full functionality: |
dependencies |
24 |
|
|
25 |
|
from CPAN or (for win32) a ppm-repository: |
26 |
|
App::Config (CPAN only) |
27 |
|
Tie::IxHash |
28 |
|
Log::Dispatch |
29 |
|
Log::Dispatch::Config |
30 |
|
mixin |
31 |
|
Hash::Merge |
32 |
|
Tie::SecureHash |
33 |
|
Date::Manip |
34 |
|
|
35 |
|
from nfo/perl/libs: |
36 |
|
Data::Compare |
37 |
Data::Storage |
Data::Storage |
38 |
Data::Transform |
Data::Transform |
39 |
Data::Compare |
DesignPattern:: (to be renamed into Class::) |
40 |
... and all their dependencies |
... and all their dependencies. |
41 |
|
|
42 |
AUTHORS / COPYRIGHT |
additional modules required for 'outlook2ldap' |
|
The Data::Storage module is Copyright (c) 2002 Andreas Motl. All rights |
|
|
reserved. |
|
43 |
|
|
44 |
You may distribute it under the terms of either the GNU General Public |
Win32::OLE |
45 |
License or the Artistic License, as specified in the Perl README file. |
Net::LDAP |
46 |
|
|
47 |
|
additional modules required for 'db2db' |
48 |
|
|
49 |
|
DBI |
50 |
|
DBD::mysql |
51 |
|
DBD::CSV |
52 |
|
Tangram |
53 |
|
|
54 |
|
AUTHORS / COPYRIGHT / LICENSE |
55 |
|
The Data::Transfer::Sync module is Copyright (c) 2002 Andreas Motl. |
56 |
|
All rights reserved. |
57 |
|
|
58 |
|
Code contributions, suggestions, support and ideas by |
59 |
|
Sebastian Utz <su@tunemedia.de> and Jan Hoffmann <jan@romain-volk.com>. |
60 |
|
|
61 |
|
You may distribute it under the terms of either the GNU General Public |
62 |
|
License or the Artistic License, as specified in the Perl README file. |
63 |
|
|
64 |
SUPPORT / WARRANTY |
SUPPORT / WARRANTY |
65 |
Data::Storage is free software. IT COMES WITHOUT WARRANTY OF ANY KIND. |
Data::Transfer::Sync is free software. IT COMES WITHOUT WARRANTY OF ANY KIND. |
66 |
|
|
67 |
|
Please feel free to contact me at <andreas.motl@ilo.de>. |
68 |
|
|
69 |
|
EXAMPLE OUTPUT CHUNKS |
70 |
|
Synchronizing orm objects against entries in a .csv file - 17 items modified |
71 |
|
|
72 |
|
notice: PULL INTO backend NODE * TYPE Country SELECT NODE * TYPE [n/a] FROM base USING MODULE BizWorks::ResourceMapping; |
73 |
|
notice: OEF::Script::AbstractFeeder: |
74 |
|
- Loading data of type [n/a] and |
75 |
|
filtered by * from the storage named base |
76 |
|
to the storage named backend - filtered by *. |
77 |
|
- Will attempt to convert the data to Country. |
78 |
|
info: Data::Transfer::Sync::API->syncNodes: source=backend/* [Country] <- target=base/* [] |
79 |
|
:c:c:c:c:c:c:c:ctr:c:c:c:c:c:c:c:ctr:c:c:c:c:c:c:ctr:c:ctr:c:c:c:c:c:c:c:c:c:c:c:ctr:c:c:c:ctr:c:c:c:c:c:c:c:ctr:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c |
80 |
|
:c:c:c:c:c:c:c:c:ctr:c:c:c:c:c:c:c:c:c:c:c:ctr:c:c:c:c:ctr:c:c:c:ctr:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:ctr:ctr:c:c:c:c:ctr:c:c:c:c:c:c:c |
81 |
|
:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:ctr:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:ctr:c:c:ctr:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c |
82 |
|
info: Data::Transfer::Sync::Core->_startsync: statistics: in_sync: 180 / total: 197 / attempt_modify: 17 / ok: 17 |
83 |
|
|
84 |
|
Synchronizing orm objects against entries in a .csv file - in-sync |
85 |
|
|
86 |
|
notice: PULL INTO backend NODE * TYPE Country SELECT NODE * TYPE [n/a] FROM base USING MODULE BizWorks::ResourceMapping; |
87 |
|
notice: OEF::Script::AbstractFeeder: |
88 |
|
- Loading data of type [n/a] and |
89 |
|
filtered by * from the storage named base |
90 |
|
to the storage named backend - filtered by *. |
91 |
|
- Will attempt to convert the data to Country. |
92 |
|
info: Data::Transfer::Sync::API->syncNodes: source=backend/* [Country] <- target=base/* [] |
93 |
|
:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c |
94 |
|
:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c |
95 |
|
:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c:c |
96 |
|
info: Data::Transfer::Sync::Core->_startsync: statistics: in_sync: 197 / total: 197 |
97 |
|
|
98 |
BUGS |
BUGS |
99 |
When in "import" mode for windows file - DBD::AutoCSV may hang. |
When in "import" mode for windows file - DBD::AutoCSV may hang. |
100 |
Hint: Maybe the source node contains an ident-, but no checksum column? |
Hint: Maybe the source node contains an ident-, but no checksum-column? |
101 |
|
|
102 |
USER LEVEL ERRORS |
USER LEVEL ERRORS |
103 |
Mapping |
Mapping |
169 |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
170 |
|
|
171 |
TODO |
TODO |
172 |
- sub _resolveIdentProvider |
o sub _resolveIdentProvider |
173 |
- wrap _doModifySource and _doTransferTarget around a core function which can change virtually any type of node |
x wrap _doModifySource and _doTransferTarget around a core function which can change virtually any type of node |
174 |
- split this module up into Sync.pm, Sync/Core.pm, Sync/Compare.pm and Sync/Compare/Checksum.pm |
x split this module up into Sync.pm, Sync/Core.pm, Sync/Compare.pm and Sync/Compare/Checksum.pm |
175 |
- introduce _compareNodes as a core method and wrap it around methods in Sync/Compare/Checksum.pm |
o introduce _compareNodes as a core method and wrap it around methods in Sync/Compare/Checksum.pm |
176 |
- introduce Sync/Compare/MyComparisonImplementation.pm |
x introduce Sync/Compare/MyComparisonImplementation.pm |
177 |
- some generic deferring method - e.g. "$self->defer(action)" - to be able to accumulate a bunch of actions for later processing |
o some generic deferring method - e.g. "$self->defer(action)" - to be able to accumulate a bunch of actions for later processing |
178 |
- this implies everything done is _really_ split up into generic actions - how else would we defer them??? |
- this implies everything done is _really_ split up into generic actions - how else would we defer them??? |
179 |
- example uses: |
- example uses: |
180 |
- fetch whole checksum list from node |
- fetch whole checksum list from node |
181 |
- remember source ident retransmits |
- remember source ident retransmits |
182 |
- remember: this is convenient - and maybe / of course faster - but we'll loose "per-node-atomic" operations |
- remember: this is convenient - and maybe / of course faster - but we'll loose "per-node-atomic" operations |
183 |
- feature: mechanism to implicit inject checksum property to nodes (alter table / modify schema) |
o feature: mechanism to implicit inject checksum property to nodes (alter table / modify schema) |
184 |
- expand statistics / keep track of: |
o expand statistics / keep track of: |
185 |
- touched/untouched nodes |
- touched/untouched nodes |
186 |
- full sync |
o full sync |
187 |
- just do a push and a pull for now but use stats for touched nodes in between to speed up things |
- just do a push and a pull for now but use stats for touched nodes in between to speed up things |
188 |
- introduce some new metadata flags for a synchronization partner which is (e.g.) of "source" or "target": |
o introduce some new metadata flags for a descent |
189 |
- isNewNodePropagator |
- isNewNodePropagator |
|
- isWriteProtected |
|
|
- add additional mechanisms: "how is the sync done exactly?" |
|
|
+ based on ident/checksum (this compares uniqueness) |
|
|
- based on ident/timestamp (this compares up-to-date-ness) |
|
|
- based on abstract rules (free, script, ..., maybe taken from a rulebase) ... |
|
|
... combining queries on metadata (e.g. flags) associated with the object (may also be stored inside the object itself for convenience) |
|
|
=> (example): |
|
|
$admin = new Person( admin => 1); |
|
|
$user1 = new Person(); |
|
|
$user2 = new Person( watched_by => $admin ); |
|
|
$context = new Context( postmaster => $admin ); |
|
|
$data = new Data( created_by => $user2, crud_hooks => { u => $admin } ); |
|
|
$admin-> |
|
190 |
|
x isWriteProtected |
191 |
|
|