Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
N
neo
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Stefane Fermigier
neo
Commits
d7c9395b
Commit
d7c9395b
authored
Jan 24, 2018
by
Kirill Smelkov
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
.
parent
6c35f692
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
3 additions
and
49 deletions
+3
-49
go/NOTES
go/NOTES
+0
-46
go/neo/proto.go
go/neo/proto.go
+1
-1
neo/storage/handlers/storage.py
neo/storage/handlers/storage.py
+2
-2
No files found.
go/NOTES
View file @
d7c9395b
...
@@ -350,52 +350,6 @@ PACKET x #0x0003 GetObject > S1 (127.0.0.1:4452
...
@@ -350,52 +350,6 @@ PACKET x #0x0003 GetObject > S1 (127.0.0.1:4452
...
...
Replication communication
-------------------------
replication types:
- asynchronous (e.g. to backup cluster or to replicate already committed data to newly added storage node)
- synchronous (on transaction commit - C stores data to all S assigned for
oid in PT and waits for all stores to complete)
XXX synchronous replication interracts with commit.
XXX "OUT_OF_DATE -> UP_TO_DATE is done in several steps" -> XXX which?
XXX
""" A replicating node can not depend on other nodes to fetch the data
recently/being committed because that can not be done atomically: it could miss
writes between the processing of its request by a source node and the reception
of the answer. """
XXX -> should not be the case with proper dump-stream based replication protocol.
XXX -> jm: document example
Therefore, outdated cells are writable: a storage node asks the master for
transactions being committed and then it is expected to fully receive from the
client any transaction that is started after this answer.
XXX -> recheck this is good idea.
R = requester
S = seeder
item = Transaction(meta) | Object(meta+data)
R -> S []items (which R already has) XXX exact message
R <- S missing-item
R <- S missing-item
...
R <- S missing-item
R <- S []items-to-delete
Py: out-of-order answers
Py: out-of-order answers
------------------------
------------------------
...
...
go/neo/proto.go
View file @
d7c9395b
...
@@ -972,7 +972,7 @@ type NotifyTransactionFinished struct {
...
@@ -972,7 +972,7 @@ type NotifyTransactionFinished struct {
type
Replicate
struct
{
type
Replicate
struct
{
Tid
zodb
.
Tid
Tid
zodb
.
Tid
UpstreamName
string
UpstreamName
string
SourceDict
map
[
uint32
/*PNumber*/
]
string
// partition -> address
SourceDict
map
[
uint32
/*PNumber*/
]
string
// partition -> address
FIXME string -> Address
}
}
// Notify the master node that a partition has been successfully replicated
// Notify the master node that a partition has been successfully replicated
...
...
neo/storage/handlers/storage.py
View file @
d7c9395b
...
@@ -219,7 +219,7 @@ class StorageOperationHandler(EventHandler):
...
@@ -219,7 +219,7 @@ class StorageOperationHandler(EventHandler):
# Sending such packet does not mark the connection
# Sending such packet does not mark the connection
# for writing if there's too little data in the buffer.
# for writing if there's too little data in the buffer.
conn
.
send
(
Packets
.
AddTransaction
(
tid
,
user
,
conn
.
send
(
Packets
.
AddTransaction
(
tid
,
user
,
desc
,
ext
,
packed
,
ttid
,
oid_list
),
msg_id
)
desc
,
ext
,
packed
,
ttid
,
oid_list
),
msg_id
)
# NOTE msg_id: ooo answer
# To avoid delaying several connections simultaneously,
# To avoid delaying several connections simultaneously,
# and also prevent the backend from scanning different
# and also prevent the backend from scanning different
# parts of the DB at the same time, we ask the
# parts of the DB at the same time, we ask the
...
@@ -269,7 +269,7 @@ class StorageOperationHandler(EventHandler):
...
@@ -269,7 +269,7 @@ class StorageOperationHandler(EventHandler):
%
partition
),
msg_id
)
%
partition
),
msg_id
)
return
return
# Same as in askFetchTransactions.
# Same as in askFetchTransactions.
conn
.
send
(
Packets
.
AddObject
(
oid
,
*
object
),
msg_id
)
conn
.
send
(
Packets
.
AddObject
(
oid
,
*
object
),
msg_id
)
# NOTE msg_id: ooo answer
yield
conn
.
buffering
yield
conn
.
buffering
conn
.
send
(
Packets
.
AnswerFetchObjects
(
conn
.
send
(
Packets
.
AnswerFetchObjects
(
pack_tid
,
next_tid
,
next_oid
,
object_dict
),
msg_id
)
# NOTE msg_id: out-of-order answer
pack_tid
,
next_tid
,
next_oid
,
object_dict
),
msg_id
)
# NOTE msg_id: out-of-order answer
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment