6 |
## |
## |
7 |
## ---------------------------------------------------------------------------------------- |
## ---------------------------------------------------------------------------------------- |
8 |
## $Log$ |
## $Log$ |
9 |
|
## Revision 1.2 2003/01/19 02:05:42 joko |
10 |
|
## - removed pod-documentation: now in Data/Transfer/Sync.pod |
11 |
|
## |
12 |
## Revision 1.1 2003/01/19 01:23:04 joko |
## Revision 1.1 2003/01/19 01:23:04 joko |
13 |
## + new from Data/Transfer/Sync.pm |
## + new from Data/Transfer/Sync.pm |
14 |
## |
## |
1207 |
$self->{meta}->{$descent}->{storage}->eraseAll($node); |
$self->{meta}->{$descent}->{storage}->eraseAll($node); |
1208 |
} |
} |
1209 |
|
|
|
|
|
|
=pod |
|
|
|
|
|
|
|
|
=head1 DESCRIPTION |
|
|
|
|
|
Data::Transfer::Sync is a module providing a generic synchronization process |
|
|
across arbitrary/multiple storages based on a ident/checksum mechanism. |
|
|
It sits on top of Data::Storage. |
|
|
|
|
|
|
|
|
=head1 REQUIREMENTS |
|
|
|
|
|
For full functionality: |
|
|
Data::Storage |
|
|
Data::Transform |
|
|
Data::Compare |
|
|
... and all their dependencies |
|
|
|
|
|
|
|
|
=head1 AUTHORS / COPYRIGHT |
|
|
|
|
|
The Data::Storage module is Copyright (c) 2002 Andreas Motl. |
|
|
All rights reserved. |
|
|
|
|
|
You may distribute it under the terms of either the GNU General Public |
|
|
License or the Artistic License, as specified in the Perl README file. |
|
|
|
|
|
|
|
|
=head1 SUPPORT / WARRANTY |
|
|
|
|
|
Data::Storage is free software. IT COMES WITHOUT WARRANTY OF ANY KIND. |
|
|
|
|
|
|
|
|
|
|
|
=head1 BUGS |
|
|
|
|
|
When in "import" mode for windows file - DBD::AutoCSV may hang. |
|
|
Hint: Maybe the source node contains an ident-, but no checksum-column? |
|
|
|
|
|
|
|
|
=head1 USER LEVEL ERRORS |
|
|
|
|
|
=head4 Mapping |
|
|
|
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - |
|
|
info: BizWorks::Process::Setup->syncResource( source_node Currency mode PULL erase 0 import 0 )critical: BizWorks::Process::Setup->startSync: Can't access mapping for node "Currency" - please check BizWorks::ResourceMapping. |
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - |
|
|
You have to create a sub for each node used in synchronization inside named Perl module. The name of this sub _must_ match |
|
|
the name of the node you want to sync. This sub holds mapping metadata to give the engine hints about how |
|
|
to access the otherwise generic nodes. |
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - |
|
|
|
|
|
|
|
|
=head4 DBD::AutoCSV's rulebase |
|
|
|
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - |
|
|
info: BizWorks::Process::Setup->syncResource( source_node Currency mode PULL erase 0 import 0 ) |
|
|
info: Data::Transfer::Sync->syncNodes: source=L/Currency <- target=R/currencies.csv |
|
|
|
|
|
Execution ERROR: Error while scanning: Missing first row or scanrule not applied. at C:/home/amo/develop/netfrag.org/nfo/perl/libs/DBD/CSV.p |
|
|
m line 165, <GEN9> line 1. |
|
|
called from C:/home/amo/develop/netfrag.org/nfo/perl/libs/Data/Storage/Handler/DBI.pm at 123. |
|
|
|
|
|
DBI-Error: DBD::AutoCSV::st fetchrow_hashref failed: Attempt to fetch row from a Non-SELECT statement |
|
|
notice: Data::Transfer::Sync->syncNodes: No nodes to synchronize. |
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - |
|
|
DBD::AutoCSV contains a rulebase which is spooled down while attempting to guess the style of the csv file regarding |
|
|
parameters like newline (eol), column-seperation-character (sep_char), quoting character (quote_char). |
|
|
If this spool runs out of entries and no style could be resolved, DBD::CSV dies causing this "Execution ERROR" which |
|
|
results in a "DBI-Error" afterwards. |
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - |
|
|
|
|
|
|
|
|
=head4 Check structure of source node |
|
|
|
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - |
|
|
info: Data::Transfer::Sync->syncNodes: source=L/Currency <- target=R/currencies.csv |
|
|
critical: Data::Transfer::Sync->syncNodes: Can not synchronize: No ident found in source node, maybe try to "import" this node first. |
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - |
|
|
If lowlevel detection succeeds, but no other required informations are found, this message is issued. |
|
|
"Other informations" might be: |
|
|
- column-header-row completely missing |
|
|
- ident column is empty |
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - |
|
|
|
|
|
|
|
|
=head4 Modify structure of source node |
|
|
|
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - |
|
|
info: Data::Transfer::Sync->syncNodes: source=L/Currency <- target=R/currencies.csv |
|
|
info: Data::Transfer::Sync->_prepareNode_MetaProperties( descent source ) |
|
|
warning: Data::Transfer::Sync->_prepareNode_MetaProperties: node is lacking meta properties - will try to alter... |
|
|
SQL ERROR: Command 'ALTER' not recognized or not supported! |
|
|
|
|
|
SQL ERROR: Command 'ALTER' not recognized or not supported! |
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - |
|
|
The Engine found a node which structure does not match the required. It tries to alter this automatically - only when doing "import" - |
|
|
but the DBD driver (in this case DBD::CSV) gets in the way croaking not to be able to do this. |
|
|
This could also appear if your database connection has insufficient rights to modify the database structure. |
|
|
DBD::CSV croaks because it doesn't implement the ALTER command, so please edit your columns manually. |
|
|
Hint: Add columns with the names of your "ident" and "checksum" property specifications. |
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - |
|
|
|
|
|
|
|
|
=head4 Load source node by ident |
|
|
|
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - |
|
|
info: Data::Transfer::Sync->_prepareNode_DummyIdent( descent source ) |
|
|
pcritical: Data::Transfer::Sync->_modifyNode failed: "source" node is empty. |
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - |
|
|
The source node could not be loaded. Maybe the ident is missing. Please check manually. |
|
|
Hint: Like above, the ident and/or checksum columns may be missing.... |
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - |
|
|
|
|
|
|
|
|
=head1 TODO |
|
|
|
|
|
- sub _resolveIdentProvider |
|
|
- wrap _doModifySource and _doTransferTarget around a core function which can change virtually any type of node |
|
|
- split this module up into Sync.pm, Sync/Core.pm, Sync/Compare.pm and Sync/Compare/Checksum.pm |
|
|
- introduce _compareNodes as a core method and wrap it around methods in Sync/Compare/Checksum.pm |
|
|
- introduce Sync/Compare/MyComparisonImplementation.pm |
|
|
- some generic deferring method - e.g. "$self->defer(action)" - to be able to accumulate a bunch of actions for later processing |
|
|
- this implies everything done is _really_ split up into generic actions - how else would we defer them??? |
|
|
- example uses: |
|
|
- fetch whole checksum list from node |
|
|
- remember source ident retransmits |
|
|
- remember: this is convenient - and maybe / of course faster - but we'll loose "per-node-atomic" operations |
|
|
- feature: mechanism to implicit inject checksum property to nodes (alter table / modify schema) |
|
|
- expand statistics / keep track of: |
|
|
- touched/untouched nodes |
|
|
- full sync |
|
|
- just do a push and a pull for now but use stats for touched nodes in between to speed up things |
|
|
- introduce some new metadata flags for a synchronization partner which is (e.g.) of "source" or "target": |
|
|
- isNewNodePropagator |
|
|
- isWriteProtected |
|
|
|
|
|
|
|
|
=cut |
|
|
|
|
1210 |
1; |
1; |