| 1 |
DESCRIPTION |
| 2 |
Data::Transfer::Sync is a module providing a generic synchronization |
| 3 |
process across arbitrary/multiple storages based on a ident/checksum |
| 4 |
mechanism. It sits on top of Data::Storage. |
| 5 |
|
| 6 |
REQUIREMENTS |
| 7 |
For full functionality: |
| 8 |
Data::Storage |
| 9 |
Data::Transform |
| 10 |
Data::Compare |
| 11 |
... and all their dependencies |
| 12 |
|
| 13 |
AUTHORS / COPYRIGHT |
| 14 |
The Data::Storage module is Copyright (c) 2002 Andreas Motl. All rights |
| 15 |
reserved. |
| 16 |
|
| 17 |
You may distribute it under the terms of either the GNU General Public |
| 18 |
License or the Artistic License, as specified in the Perl README file. |
| 19 |
|
| 20 |
SUPPORT / WARRANTY |
| 21 |
Data::Storage is free software. IT COMES WITHOUT WARRANTY OF ANY KIND. |
| 22 |
|
| 23 |
BUGS |
| 24 |
When in "import" mode for windows file - DBD::AutoCSV may hang. |
| 25 |
Hint: Maybe the source node contains an ident-, but no checksum column? |
| 26 |
|
| 27 |
USER LEVEL ERRORS |
| 28 |
Mapping |
| 29 |
|
| 30 |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
| 31 |
info: BizWorks::Process::Setup->syncResource( source_node Currency mode PULL erase 0 import 0 )critical: BizWorks::Process::Setup->startSync: Can't access mapping for node "Currency" - please check BizWorks::ResourceMapping. |
| 32 |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
| 33 |
You have to create a sub for each node used in synchronization inside named Perl module. The name of this sub _must_ match |
| 34 |
the name of the node you want to sync. This sub holds mapping metadata to give the engine hints about how |
| 35 |
to access the otherwise generic nodes. |
| 36 |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
| 37 |
|
| 38 |
DBD::AutoCSV's rulebase |
| 39 |
|
| 40 |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
| 41 |
info: BizWorks::Process::Setup->syncResource( source_node Currency mode PULL erase 0 import 0 ) |
| 42 |
info: Data::Transfer::Sync->syncNodes: source=L/Currency <- target=R/currencies.csv |
| 43 |
|
| 44 |
Execution ERROR: Error while scanning: Missing first row or scanrule not applied. at C:/home/amo/develop/netfrag.org/nfo/perl/libs/DBD/CSV.p |
| 45 |
m line 165, <GEN9> line 1. |
| 46 |
called from C:/home/amo/develop/netfrag.org/nfo/perl/libs/Data/Storage/Handler/DBI.pm at 123. |
| 47 |
|
| 48 |
DBI-Error: DBD::AutoCSV::st fetchrow_hashref failed: Attempt to fetch row from a Non-SELECT statement |
| 49 |
notice: Data::Transfer::Sync->syncNodes: No nodes to synchronize. |
| 50 |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
| 51 |
DBD::AutoCSV contains a rulebase which is spooled down while attempting to guess the style of the csv file regarding |
| 52 |
parameters like newline (eol), column-seperation-character (sep_char), quoting character (quote_char). |
| 53 |
If this spool runs out of entries and no style could be resolved, DBD::CSV dies causing this "Execution ERROR" which |
| 54 |
results in a "DBI-Error" afterwards. |
| 55 |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
| 56 |
|
| 57 |
Check structure of source node |
| 58 |
|
| 59 |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
| 60 |
info: Data::Transfer::Sync->syncNodes: source=L/Currency <- target=R/currencies.csv |
| 61 |
critical: Data::Transfer::Sync->syncNodes: Can not synchronize: No ident found in source node, maybe try to "import" this node first. |
| 62 |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
| 63 |
If lowlevel detection succeeds, but no other required informations are found, this message is issued. |
| 64 |
"Other informations" might be: |
| 65 |
- column-header-row completely missing |
| 66 |
- ident column is empty |
| 67 |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
| 68 |
|
| 69 |
Modify structure of source node |
| 70 |
|
| 71 |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
| 72 |
info: Data::Transfer::Sync->syncNodes: source=L/Currency <- target=R/currencies.csv |
| 73 |
info: Data::Transfer::Sync->_prepareNode_MetaProperties( descent source ) |
| 74 |
warning: Data::Transfer::Sync->_prepareNode_MetaProperties: node is lacking meta properties - will try to alter... |
| 75 |
SQL ERROR: Command 'ALTER' not recognized or not supported! |
| 76 |
|
| 77 |
SQL ERROR: Command 'ALTER' not recognized or not supported! |
| 78 |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
| 79 |
The Engine found a node which structure does not match the required. It tries to alter this automatically - only when doing "import" - |
| 80 |
but the DBD driver (in this case DBD::CSV) gets in the way croaking not to be able to do this. |
| 81 |
This could also appear if your database connection has insufficient rights to modify the database structure. |
| 82 |
DBD::CSV croaks because it doesn't implement the ALTER command, so please edit your columns manually. |
| 83 |
Hint: Add columns with the names of your "ident" and "checksum" property specifications. |
| 84 |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
| 85 |
|
| 86 |
Load source node by ident |
| 87 |
|
| 88 |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
| 89 |
info: Data::Transfer::Sync->_prepareNode_DummyIdent( descent source ) |
| 90 |
pcritical: Data::Transfer::Sync->_modifyNode failed: "source" node is empty. |
| 91 |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
| 92 |
The source node could not be loaded. Maybe the ident is missing. Please check manually. |
| 93 |
Hint: Like above, the ident and/or checksum columns may be missing.... |
| 94 |
- - - - - - - - - - - - - - - - - - - - - - - - - - |
| 95 |
|
| 96 |
TODO |
| 97 |
- sub _resolveIdentProvider |
| 98 |
- wrap _doModifySource and _doTransferTarget around a core function which can change virtually any type of node |
| 99 |
- split this module up into Sync.pm, Sync/Core.pm, Sync/Compare.pm and Sync/Compare/Checksum.pm |
| 100 |
- introduce _compareNodes as a core method and wrap it around methods in Sync/Compare/Checksum.pm |
| 101 |
- introduce Sync/Compare/MyComparisonImplementation.pm |
| 102 |
- some generic deferring method - e.g. "$self->defer(action)" - to be able to accumulate a bunch of actions for later processing |
| 103 |
- this implies everything done is _really_ split up into generic actions - how else would we defer them??? |
| 104 |
- example uses: |
| 105 |
- fetch whole checksum list from node |
| 106 |
- remember source ident retransmits |
| 107 |
- remember: this is convenient - and maybe / of course faster - but we'll loose "per-node-atomic" operations |
| 108 |
- feature: mechanism to implicit inject checksum property to nodes (alter table / modify schema) |
| 109 |
- expand statistics / keep track of: |
| 110 |
- touched/untouched nodes |
| 111 |
- full sync |
| 112 |
- just do a push and a pull for now but use stats for touched nodes in between to speed up things |
| 113 |
- introduce some new metadata flags for a synchronization partner which is (e.g.) of "source" or "target": |
| 114 |
- isNewNodePropagator |
| 115 |
- isWriteProtected |
| 116 |
|
Parent Directory
| 
Revision Log
