71 |
|
|
72 |
=head1 AUTHORS / COPYRIGHT / LICENSE |
=head1 AUTHORS / COPYRIGHT / LICENSE |
73 |
|
|
74 |
The Data::Transfer::Sync module is Copyright (c) 2002 Andreas Motl. |
The Data::Transfer::Sync module is Copyright (c) 2002, 2003 Andreas Motl. |
75 |
All rights reserved. |
All Rights Reserved. |
76 |
|
|
77 |
Code contributions, suggestions, support and ideas by |
Code contributions, suggestions, support and ideas by |
78 |
Sebastian Utz <su@tunemedia.de> and Jan Hoffmann <jan@romain-volk.com>. |
Sebastian Utz <su@tunemedia.de> and Jan Hoffmann <janosch@ultrajan.de>. |
79 |
|
|
80 |
You may distribute it under the terms of either the GNU General Public |
You may distribute it under the terms of either the GNU General Public |
81 |
License or the Artistic License, as specified in the Perl README file. |
License or the Artistic License, as specified in the Perl README file. |
82 |
|
|
83 |
|
|
84 |
|
=head1 COPYRIGHT NOTE |
85 |
|
|
86 |
|
* CAESAR - Collaborative And Easy System ARchitecture |
87 |
|
* |
88 |
|
* Data Synchronization Module |
89 |
|
* |
90 |
|
* Permission to use, copy, modify and distribute this software and its |
91 |
|
* documentation is hereby granted, provided that both the copyright |
92 |
|
* notice and this permission notice appear in all copies of the |
93 |
|
* software, derivative works or modified versions, and any portions |
94 |
|
* thereof, and that both notices appear in supporting documentation. |
95 |
|
* |
96 |
|
* NETFRAG.ORG AND THE AUTHOR ALLOW FREE USE OF THIS SOFTWARE |
97 |
|
* IN ITS "AS IS" CONDITION. WE DISCLAIM ANY LIABILITY OF ANY KIND |
98 |
|
* FOR ANY DAMAGES WHATSOEVER RESULTING FROM THE USE OF THIS SOFTWARE. |
99 |
|
|
100 |
|
|
101 |
|
|
102 |
=head1 SUPPORT / WARRANTY |
=head1 SUPPORT / WARRANTY |
103 |
|
|
104 |
Data::Transfer::Sync is free software. IT COMES WITHOUT WARRANTY OF ANY KIND. |
Data::Transfer::Sync is free software. IT COMES WITHOUT WARRANTY OF ANY KIND. |
105 |
|
|
106 |
Please feel free to contact me at <andreas.motl@ilo.de>. |
Please feel free to contact me at <andreas.motl@ilo.de>. |
107 |
|
|
108 |
|
Any improvements and contributions are greatly appreciated. |
109 |
|
|
110 |
|
We have set up a newsgroup. (currently not much traffic) |
111 |
|
Be welcome to visit news://news.netfrag.org/nfo.caesar or - if your system |
112 |
|
lacks a newsreader - try http://news.netfrag.org/nfo.caesar. |
113 |
|
|
114 |
|
|
115 |
|
|
116 |
|
|
117 |
|
=head1 TODO |
118 |
|
|
119 |
|
=head4 overall |
120 |
|
|
121 |
|
o try to get this stuff together with SyncML (http://syncml.org/) somehow ...? |
122 |
|
|
123 |
|
|
124 |
|
=head4 this code |
125 |
|
|
126 |
|
o sub _resolveIdentProvider |
127 |
|
x wrap _doModifySource and _doTransferTarget around a core function which can change virtually any type of node |
128 |
|
x split this module up into Sync.pm, Sync/Core.pm, Sync/Compare.pm and Sync/Compare/Checksum.pm |
129 |
|
o introduce _compareNodes as a core method and wrap it around methods in Sync/Compare/Checksum.pm |
130 |
|
x introduce Sync/Compare/MyComparisonImplementation.pm |
131 |
|
o some generic deferring method - e.g. "$self->defer(action)" - to be able to accumulate a bunch of actions for later processing |
132 |
|
- this implies everything done is _really_ split up into generic actions - how else would we defer them??? |
133 |
|
- example uses: |
134 |
|
- fetch whole checksum list from node |
135 |
|
- remember source ident retransmits |
136 |
|
- remember: this is convenient - and maybe / of course faster - but we'll loose "per-node-atomic" operations |
137 |
|
o feature: mechanism to implicit inject checksum property to nodes (alter table / modify schema) |
138 |
|
o expand statistics / keep track of: |
139 |
|
- touched/untouched nodes |
140 |
|
o full sync |
141 |
|
- just do a push and a pull for now but use stats for touched nodes in between to speed up things |
142 |
|
o introduce some new metadata flags for a descent |
143 |
|
- isNewNodePropagator |
144 |
|
x isWriteProtected |
145 |
|
|
146 |
|
|
147 |
|
|
149 |
|
|
150 |
=head2 Synchronizing orm objects against entries in a .csv file - 17 items modified |
=head2 Synchronizing orm objects against entries in a .csv file - 17 items modified |
151 |
|
|
152 |
notice: PULL INTO backend NODE * TYPE Country SELECT NODE * TYPE [n/a] FROM base USING MODULE BizWorks::ResourceMapping; |
notice: PULL INTO backend NODE * TYPE Country SELECT NODE * TYPE [n/a] FROM base USING MODULE MyApp::ResourceMapping; |
153 |
notice: OEF::Script::AbstractFeeder: |
notice: OEF::Script::AbstractFeeder: |
154 |
- Loading data of type [n/a] and |
- Loading data of type [n/a] and |
155 |
filtered by * from the storage named base |
filtered by * from the storage named base |
163 |
|
|
164 |
=head2 Synchronizing orm objects against entries in a .csv file - in-sync |
=head2 Synchronizing orm objects against entries in a .csv file - in-sync |
165 |
|
|
166 |
notice: PULL INTO backend NODE * TYPE Country SELECT NODE * TYPE [n/a] FROM base USING MODULE BizWorks::ResourceMapping; |
notice: PULL INTO backend NODE * TYPE Country SELECT NODE * TYPE [n/a] FROM base USING MODULE MyApp::ResourceMapping; |
167 |
notice: OEF::Script::AbstractFeeder: |
notice: OEF::Script::AbstractFeeder: |
168 |
- Loading data of type [n/a] and |
- Loading data of type [n/a] and |
169 |
filtered by * from the storage named base |
filtered by * from the storage named base |
188 |
=head4 Mapping |
=head4 Mapping |
189 |
|
|
190 |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
191 |
info: BizWorks::Process::Setup->syncResource( source_node Currency mode PULL erase 0 import 0 )critical: BizWorks::Process::Setup->startSync: Can't access mapping for node "Currency" - please check BizWorks::ResourceMapping. |
info: MyApp::Process::Setup->syncResource( source_node Currency mode PULL erase 0 import 0 )critical: MyApp::Process::Setup->startSync: Can't access mapping for node "Currency" - please check MyApp::ResourceMapping. |
192 |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
193 |
You have to create a sub for each node used in synchronization inside named Perl module. The name of this sub _must_ match |
You have to create a sub for each node used in synchronization inside named Perl module. The name of this sub _must_ match |
194 |
the name of the node you want to sync. This sub holds mapping metadata to give the engine hints about how |
the name of the node you want to sync. This sub holds mapping metadata to give the engine hints about how |
199 |
=head4 DBD::AutoCSV's rulebase |
=head4 DBD::AutoCSV's rulebase |
200 |
|
|
201 |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
202 |
info: BizWorks::Process::Setup->syncResource( source_node Currency mode PULL erase 0 import 0 ) |
info: MyApp::Process::Setup->syncResource( source_node Currency mode PULL erase 0 import 0 ) |
203 |
info: Data::Transfer::Sync->syncNodes: source=L/Currency <- target=R/currencies.csv |
info: Data::Transfer::Sync->syncNodes: source=L/Currency <- target=R/currencies.csv |
204 |
|
|
205 |
Execution ERROR: Error while scanning: Missing first row or scanrule not applied. at C:/home/amo/develop/netfrag.org/nfo/perl/libs/DBD/CSV.p |
Execution ERROR: Error while scanning: Missing first row or scanrule not applied. at C:/home/amo/develop/netfrag.org/nfo/perl/libs/DBD/CSV.p |
258 |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
259 |
|
|
260 |
|
|
|
=head1 TODO |
|
|
|
|
|
o sub _resolveIdentProvider |
|
|
x wrap _doModifySource and _doTransferTarget around a core function which can change virtually any type of node |
|
|
x split this module up into Sync.pm, Sync/Core.pm, Sync/Compare.pm and Sync/Compare/Checksum.pm |
|
|
o introduce _compareNodes as a core method and wrap it around methods in Sync/Compare/Checksum.pm |
|
|
x introduce Sync/Compare/MyComparisonImplementation.pm |
|
|
o some generic deferring method - e.g. "$self->defer(action)" - to be able to accumulate a bunch of actions for later processing |
|
|
- this implies everything done is _really_ split up into generic actions - how else would we defer them??? |
|
|
- example uses: |
|
|
- fetch whole checksum list from node |
|
|
- remember source ident retransmits |
|
|
- remember: this is convenient - and maybe / of course faster - but we'll loose "per-node-atomic" operations |
|
|
o feature: mechanism to implicit inject checksum property to nodes (alter table / modify schema) |
|
|
o expand statistics / keep track of: |
|
|
- touched/untouched nodes |
|
|
o full sync |
|
|
- just do a push and a pull for now but use stats for touched nodes in between to speed up things |
|
|
o introduce some new metadata flags for a descent |
|
|
- isNewNodePropagator |
|
|
x isWriteProtected |
|
261 |
|
|
262 |
|
|
263 |
=cut |
=cut |