Package org.eclipse.jgit.internal.ketch

Class KetchReplica

java.lang.Object

org.eclipse.jgit.internal.ketch.KetchReplica

Direct Known Subclasses:

LocalReplica, RemoteGitReplica

public abstract class KetchReplica
extends java.lang.Object

A Ketch replica, either LocalReplica or RemoteGitReplica.

Replicas can be either a stock Git replica, or a Ketch-aware replica.

A stock Git replica has no special knowledge of Ketch and simply stores objects and references. Ketch communicates with the stock Git replica using the Git push wire protocol. The KetchLeader commits an agreed upon state by pushing all references to the Git replica, for example "refs/heads/master" is pushed during commit. Stock Git replicas use KetchReplica.CommitMethod.ALL_REFS to record the final state.

Ketch-aware replicas understand the RefTree sent during the proposal and during commit are able to update their own reference space to match the state represented by the RefTree. Ketch-aware replicas typically use a RefTreeDatabase and KetchReplica.CommitMethod.TXN_COMMITTED to record the final state.

KetchReplica instances are tightly coupled with a single KetchLeader. Some state may be accessed by the leader thread and uses the leader's own KetchLeader.lock to protect shared data.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	KetchReplica.CommitMethod	How this replica wants to receive Ketch commit operations.
static class	KetchReplica.CommitSpeed	Delay before committing to a replica.
static class	KetchReplica.Participation	Participation of a replica in establishing consensus.
static class	KetchReplica.State	Current state of a replica.
(package private) static class	KetchReplica.WeakRetryPush	Weakly holds a retrying replica, allowing it to garbage collect.

Field Summary

Fields

Modifier and Type	Field	Description
private KetchReplica.CommitMethod	commitMethod
private KetchReplica.CommitSpeed	commitSpeed
private java.lang.String	error
private long	lastRetryMillis
private KetchLeader	leader
(package private) static org.slf4j.Logger	log
private long	maxRetryMillis
private long	minRetryMillis
private KetchReplica.Participation	participation
private static byte[]	PEEL
private java.util.List<ReplicaPushRequest>	queued
private java.lang.String	replicaName
private long	retryAtMillis
private java.util.concurrent.Future<?>	retryFuture	Scheduled retry due to communication failure.
private java.util.Map<java.lang.String,ReceiveCommand>	running
private java.util.Map<ObjectId,java.util.List<ReceiveCommand>>	staged
private KetchReplica.State	state	What is happening with this replica.
private ObjectId	txnAccepted	Value known for "refs/txn/accepted".
private ObjectId	txnCommitted	Value known for "refs/txn/committed".
private java.util.Map<java.lang.String,ReceiveCommand>	waiting

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	KetchReplica(KetchLeader leader, java.lang.String name, ReplicaConfig cfg)	Configure a replica representation.

Method Summary

Modifier and Type	Method	Description
(package private) void	afterPush(Repository repo, ReplicaPushRequest req)	Callback from ReplicaPushRequest upon success or failure.
protected abstract void	blockingFetch(Repository repo, ReplicaFetchRequest req)	Fetch objects from the remote using the calling thread.
(package private) boolean	canDelete(Ref ref)
private static ReceiveCommand	copy(ReceiveCommand c)
private boolean	defer(ReplicaPushRequest req)
private static void	delete(java.util.List<ReceiveCommand> cmds, java.util.List<ReceiveCommand> createCmds)
protected java.lang.String	describeForLog()	Get description of this replica for error/debug logging purposes.
private void	doRetryPush()
private static boolean	equals(ObjectId a, LogIndex b)
KetchReplica.CommitMethod	getCommitMethod()	Get how Ketch will commit to the repository.
KetchReplica.CommitSpeed	getCommitSpeed()	Get when Ketch will commit to the repository.
(package private) static ObjectId	getId(Ref ref)
KetchLeader	getLeader()	Get leader instance this replica follows.
java.lang.String	getName()	Get unique-ish name for debugging.
KetchReplica.Participation	getParticipation()	Get how the replica participates in this Ketch system.
KetchSystem	getSystem()	Get system configuration.
(package private) ObjectId	getTxnAccepted()
(package private) boolean	hasAccepted(LogIndex id)
(package private) void	initialize(java.util.Map<java.lang.String,Ref> refs)	Update the leader's view of the replica after a poll.
protected java.util.Collection<ReceiveCommand>	prepareCommit(Repository git, java.util.Map<java.lang.String,Ref> current, ObjectId committed)	Build a list of commands to commit KetchReplica.CommitMethod.ALL_REFS.
private void	prepareTxnCommitted(java.util.List<ReceiveCommand> cmds, ObjectId committed)
private void	pushAsync(ReplicaPushRequest req)
(package private) void	pushCommitAsync(LogIndex committed)
(package private) void	pushTxnAcceptedAsync(Round round)	Schedule a proposal round with the replica.
private static ObjectId	readId(ReplicaPushRequest req, ReceiveCommand cmd)
private void	removeStaged(java.util.List<ReceiveCommand> cmds, ObjectId committed)
private void	retryLater(ReplicaPushRequest req)
private void	runNextPushRequest()	Determine the next push for this replica (if any) and start it.
(package private) boolean	shouldPushUnbatchedCommit(LogIndex committed, boolean leaderIdle)
protected void	shutdown()	Called by leader to perform graceful shutdown.
(package private) ReplicaSnapshot	snapshot()
protected abstract void	startPush(ReplicaPushRequest req)	Begin executing a single push.
private void	updateView(ReplicaPushRequest req, ObjectId acceptId, ReceiveCommand commitCmd)
private boolean	waitingForRetry()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

static final org.slf4j.Logger log

PEEL

private static final byte[] PEEL

leader

private final KetchLeader leader

replicaName

private final java.lang.String replicaName

participation

private final KetchReplica.Participation participation

commitMethod

private final KetchReplica.CommitMethod commitMethod

commitSpeed

private final KetchReplica.CommitSpeed commitSpeed

minRetryMillis

private final long minRetryMillis

maxRetryMillis

private final long maxRetryMillis

staged

private final java.util.Map<ObjectId,java.util.List<ReceiveCommand>> staged

running

private final java.util.Map<java.lang.String,ReceiveCommand> running

waiting

private final java.util.Map<java.lang.String,ReceiveCommand> waiting

queued

private final java.util.List<ReplicaPushRequest> queued

txnAccepted

private ObjectId txnAccepted

Value known for "refs/txn/accepted".

Raft literature refers to this as matchIndex.

txnCommitted

private ObjectId txnCommitted

Value known for "refs/txn/committed".

Raft literature refers to this as commitIndex. In traditional Raft this is a state variable inside the follower implementation, but Ketch keeps it in the leader.

state

private KetchReplica.State state

What is happening with this replica.

error

private java.lang.String error

retryFuture

private java.util.concurrent.Future<?> retryFuture

Scheduled retry due to communication failure.

lastRetryMillis

private long lastRetryMillis

retryAtMillis

private long retryAtMillis

Constructor Detail

KetchReplica

protected KetchReplica(KetchLeader leader,
                       java.lang.String name,
                       ReplicaConfig cfg)

Configure a replica representation.

Parameters:

leader - instance this replica follows.

name - unique-ish name identifying this replica for debugging.

cfg - how Ketch should treat the replica.

Method Detail

getSystem

public KetchSystem getSystem()

Get system configuration.

Returns:

system configuration.

getLeader

public KetchLeader getLeader()

Get leader instance this replica follows.

Returns:

leader instance this replica follows.

getName

public java.lang.String getName()

Get unique-ish name for debugging.

Returns:

unique-ish name for debugging.

describeForLog

protected java.lang.String describeForLog()

Get description of this replica for error/debug logging purposes.

Returns:

description of this replica for error/debug logging purposes.

getParticipation

public KetchReplica.Participation getParticipation()

Get how the replica participates in this Ketch system.

Returns:

how the replica participates in this Ketch system.

getCommitMethod

public KetchReplica.CommitMethod getCommitMethod()

Get how Ketch will commit to the repository.

Returns:

how Ketch will commit to the repository.

getCommitSpeed

public KetchReplica.CommitSpeed getCommitSpeed()

Get when Ketch will commit to the repository.

Returns:

when Ketch will commit to the repository.

shutdown

protected void shutdown()

Called by leader to perform graceful shutdown.

Default implementation cancels any scheduled retry. Subclasses may add additional logic before or after calling super.shutdown().

Called with KetchLeader.lock held by caller.

snapshot

ReplicaSnapshot snapshot()

initialize

void initialize(java.util.Map<java.lang.String,Ref> refs)

Update the leader's view of the replica after a poll.

Called with KetchLeader.lock held by caller.

Parameters:

refs - map of refs from the replica.

getTxnAccepted

ObjectId getTxnAccepted()

hasAccepted

boolean hasAccepted(LogIndex id)

equals

private static boolean equals(@Nullable
                              ObjectId a,
                              LogIndex b)

pushTxnAcceptedAsync

void pushTxnAcceptedAsync(Round round)

Schedule a proposal round with the replica.

Called with KetchLeader.lock held by caller.

Parameters:

round - current round being run by the leader.

copy

private static ReceiveCommand copy(ReceiveCommand c)

shouldPushUnbatchedCommit

boolean shouldPushUnbatchedCommit(LogIndex committed,
                                  boolean leaderIdle)

pushCommitAsync

void pushCommitAsync(LogIndex committed)

prepareTxnCommitted

private void prepareTxnCommitted(java.util.List<ReceiveCommand> cmds,
                                 ObjectId committed)

removeStaged

private void removeStaged(java.util.List<ReceiveCommand> cmds,
                          ObjectId committed)

delete

private static void delete(java.util.List<ReceiveCommand> cmds,
                           java.util.List<ReceiveCommand> createCmds)

runNextPushRequest

private void runNextPushRequest()

Determine the next push for this replica (if any) and start it.

If the replica has successfully accepted the committed state of the leader, this method will push all references to the replica using the configured KetchReplica.CommitMethod.

If the replica is KetchReplica.State.LAGGING this method will begin catch up by sending a more recent refs/txn/accepted.

Must be invoked with KetchLeader.lock held by caller.

pushAsync

private void pushAsync(ReplicaPushRequest req)

defer

private boolean defer(ReplicaPushRequest req)

waitingForRetry

private boolean waitingForRetry()

retryLater

private void retryLater(ReplicaPushRequest req)

doRetryPush

private void doRetryPush()

startPush

protected abstract void startPush(ReplicaPushRequest req)

Begin executing a single push.

This method must move processing onto another thread. Called with KetchLeader.lock held by caller.

Parameters:

req - the request to send to the replica.

afterPush

void afterPush(@Nullable
               Repository repo,
               ReplicaPushRequest req)

Callback from ReplicaPushRequest upon success or failure.

Acquires the KetchLeader.lock and updates the leader's internal knowledge about this replica to reflect what has been learned during a push to the replica. In some cases of divergence this method may take some time to determine how the replica has diverged; to reduce contention this is evaluated before acquiring the leader lock.

Parameters:

repo - local repository instance used by the push thread.

req - push request just attempted.

updateView

private void updateView(ReplicaPushRequest req,
                        @Nullable
                        ObjectId acceptId,
                        ReceiveCommand commitCmd)

readId

@Nullable
private static ObjectId readId(ReplicaPushRequest req,
                               @Nullable
                               ReceiveCommand cmd)

blockingFetch

protected abstract void blockingFetch(Repository repo,
                                      ReplicaFetchRequest req)
                               throws java.io.IOException

Fetch objects from the remote using the calling thread.

Called without KetchLeader.lock.

Parameters:

repo - local repository to fetch objects into.

req - the request to fetch from a replica.

Throws:

java.io.IOException - communication with the replica was not possible.

prepareCommit

protected java.util.Collection<ReceiveCommand> prepareCommit(Repository git,
                                                             java.util.Map<java.lang.String,Ref> current,
                                                             ObjectId committed)
                                                      throws java.io.IOException

Build a list of commands to commit KetchReplica.CommitMethod.ALL_REFS.

Parameters:

git - local leader repository to read committed state from.

current - all known references in the replica's repository. Typically this comes from a push advertisement.

committed - state being pushed to refs/txn/committed.

Returns:

commands to update during commit.

Throws:

java.io.IOException - cannot read the committed state.

canDelete

boolean canDelete(Ref ref)

getId

@NonNull
static ObjectId getId(@Nullable
                      Ref ref)