Add seed restore until section and remove 5.x seed provider info #2002

jackwaudby · 2024-12-06T12:55:07Z

Remove 5.x specific documentation from 'Seed from URI' section
Add section on seedRestoreUntil option

modules/ROOT/pages/clustering/databases.adoc

Co-authored-by: NataliaIvakina <[email protected]>

jackwaudby · 2024-12-19T09:59:03Z

@NataliaIvakina the seed restore functionality/syntax will only be available in Cypher 25 which I believe won't be available until at least 2025.02? In which case how is best to proceed here, delay merging until 2025.02? As a side question do we have to clarify whether each section/functionality/example is compatible with Cypher 25 and/or Cypher 5?

NataliaIvakina · 2024-12-19T10:54:19Z

@NataliaIvakina the seed restore functionality/syntax will only be available in Cypher 25 which I believe won't be available until at least 2025.02? In which case how is best to proceed here, delay merging until 2025.02? As a side question do we have to clarify whether each section/functionality/example is compatible with Cypher 25 and/or Cypher 5?

@jackwaudby, these are very good questions! I didn't know that Cypher 25 will be released later than Neo4j 2025.01. It complicates things.

Are you talking about the 'seedRestoreUntil' option? This option will be available only in Cypher 25, won't it?
If so, I think it makes no sense to merge the PR with the 2025.01 release if functionality is not available. Let's wait till 2025.02 and Cypher 25 to be released.

I suppose if a feature is available only in Cypher 25, we have to label it accordingly.
Do I understand correctly that this functionality is not available in Cypher 5? If so, we have to put a label, e.g., 'Not available in Cypher 5'.

jackwaudby · 2024-12-19T11:17:48Z

I didn't know that Cypher 25 will be released later than Neo4j 2025.01.

Neither did I, I only found out yesterday (apparently it could still be later than 2025.01 as well). TBH somebody probably did tell me before but I wasn't listening 😆

Are you talking about the 'seedRestoreUntil' option? This option will be available only in Cypher 25, won't it?
If so, I think it makes no sense to merge the PR with the 2025.01 release if functionality is not available. Let's wait till 2025.02 and Cypher 25 to be released.

Yes, sounds good. Let's wait until 2025.02.

Do I understand correctly that this functionality is not available in Cypher 5?

Yes correct, this is not available in Cypher 5.

modules/ROOT/pages/clustering/databases.adoc

NataliaIvakina · 2024-12-23T09:31:43Z

modules/ROOT/pages/clustering/databases.adoc

@@ -573,43 +572,29 @@ CREATE DATABASE foo OPTIONS { existingData: 'use', seedURI: 'azb://myStorageAcco
 ======
 =====

-[[s3-seed-provider]]


the same question. Is this option removed? In which Neo4j and Cypher versions?

From Neo4j 2025.01 s3 is supported only with the CloudSeedProvider. This is true for both Cypher 5 and Cypher 25 running on 2025.01.

However, in theory it would work if a user run Cypher 5 on 2025.01 and used the S3SeedProvider but it is not the approach we want to encourage.

I think there is a balancing act to be done here. The S3SeedProvider still exists in order to support the Cypher 5 usage of it. So it needs to remain in the docs, but be clearly 'Cypher 5 only'.

Suggested change to the setting description to capture this: https://github.com/neo-technology/neo4j/pull/28957

modules/ROOT/pages/clustering/databases.adoc

NataliaIvakina · 2025-01-08T11:51:33Z

modules/ROOT/pages/clustering/databases.adoc


+This will seed the database with transactions up to, but not including transaction 123.



Suggested change

[role=label--deprecated label--available-Cypher5]

[[s3-seed-provider]]

==== S3SeedProvider

The `S3SeedProvider` supports:

** `s3:`

[NOTE]

====

Neo4j 5 comes bundled with necessary libraries for AWS S3 connectivity.

Therefore, if you use `S3SeedProvider`,`aws cli` is not required but can be used with the `CloudSeedProvider`.

====

The `S3SeedProvider` requires additional configuration.

This is specified with the `seedConfig` option.

This option expects a comma-separated list of configurations.

Each configuration value is specified as a name followed by `=` and the value, as such:

[source, cypher, role="noplay"]

----

CREATE DATABASE foo OPTIONS { existingData: 'use', seedURI: 's3:/myBucket/myBackup.backup', seedConfig: 'region=eu-west-1' }

----

`S3SeedProvider` also requires passing in credentials.

These are specified with the `seedCredentials` option.

Seed credentials are securely passed from the Cypher command to each server hosting the database.

For this to work, Neo4j on each server in the cluster must be configured with identical keystores.

This is identical to the configuration required by remote aliases, see xref:database-administration/aliases/remote-database-alias-configuration.adoc#remote-alias-config-DBMS_admin-A[Configuration of DBMS with remote database alias].

If this configuration is not performed, the `seedCredentials` option fails.

[source, cypher, role="noplay"]

----

CREATE DATABASE foo OPTIONS { existingData: 'use', seedURI: 's3:/myBucket/myBackup.backup', seedConfig: 'region=eu-west-1', seedCredentials: [accessKey];[secretKey] }

----

Where `accessKey` and `secretKey` are provided by AWS.

modules/ROOT/pages/clustering/databases.adoc

nick-giles-neo · 2025-01-08T14:54:12Z

modules/ROOT/pages/clustering/databases.adoc

+The `CloudSeedProvider` supports seeding up to a specific date or transaction ID using the `seedRestoreUntil` option.

+==== Seed up to a specific date

-[NOTE]
-====
-Neo4j 5 comes bundled with necessary libraries for AWS S3 connectivity.
-Therefore, if you use `S3SeedProvider`,`aws cli` is not required but can be used with the `CloudSeedProvider`.
-====
+To seed up to a specific date, you need to pass the differential backup, which contains the data up to that date.

-The `S3SeedProvider` requires additional configuration.
-This is specified with the `seedConfig` option.
-This option expects a comma-separated list of configurations.
-Each configuration value is specified as a name followed by `=` and the value, as such:

-[source, cypher, role="noplay"]
+[source,shell]
 ----
-CREATE DATABASE foo OPTIONS { existingData: 'use', seedURI: 's3:/myBucket/myBackup.backup', seedConfig: 'region=eu-west-1' }
+CREATE DATABASE foo OPTIONS { existingData: 'use', seedURI: 's3:/myBucket/myBackup.backup', seedRestoreUntil: datetime("2019-06-01T18:40:32.142+0100") }
 ----

-`S3SeedProvider` also requires passing in credentials.
-These are specified with the `seedCredentials` option.
-Seed credentials are securely passed from the Cypher command to each server hosting the database.
-For this to work, Neo4j on each server in the cluster must be configured with identical keystores.
-This is identical to the configuration required by remote aliases, see xref:database-administration/aliases/remote-database-alias-configuration.adoc#remote-alias-config-DBMS_admin-A[Configuration of DBMS with remote database alias].
-If this configuration is not performed, the `seedCredentials` option fails.
+This will seed the database with transactions committed before the provided timestamp.


The 'seed up to date' functionality is not properly present in 2025.01, so this section needs to be removed. It works to seed up to a transaction id, but Cypher Operations need to make a change to allow creating a datetime with system as the current database to make this work nicely.

Wait, late breaking news, they merged the change before cut-off! https://github.com/neo-technology/neo4j/pull/28684

So this section about restore until datetime should remain in the doc.

neo-technology-commit-status-publisher · 2025-01-10T08:34:07Z

This PR includes documentation updates
View the updated docs at https://neo4j-docs-operations-2002.surge.sh

Updated pages:

Managing databases in a cluster

jackwaudby added 2 commits December 6, 2024 12:36

Remove 5.x series deprecations and introductions

ab62614

Add seedRestoreUntil

c5628fd

jackwaudby added dev 2025.01 labels Dec 6, 2024

NataliaIvakina self-assigned this Dec 10, 2024

renetapopova added the clustering label Dec 11, 2024

NataliaIvakina self-requested a review December 19, 2024 07:55

NataliaIvakina reviewed Dec 19, 2024

View reviewed changes

jackwaudby and others added 4 commits December 19, 2024 09:33

Update modules/ROOT/pages/clustering/databases.adoc

4e0ef85

Co-authored-by: NataliaIvakina <[email protected]>

Update modules/ROOT/pages/clustering/databases.adoc

256ae75

Co-authored-by: NataliaIvakina <[email protected]>

Update modules/ROOT/pages/clustering/databases.adoc

f0ca47e

Co-authored-by: NataliaIvakina <[email protected]>

Remove labels

9abb963

NataliaIvakina added the Cypher 25 label Dec 19, 2024

NataliaIvakina added 2025.02 and removed 2025.01 2025.02 labels Dec 19, 2024

NataliaIvakina reviewed Dec 23, 2024

View reviewed changes

jackwaudby changed the title ~~Add seed restore until section~~ Add seed restore until section and remove 5.x seed provider info Jan 3, 2025

Merge branch 'dev' into 2024-12-seed-restore-until

a36ae2d

NataliaIvakina reviewed Jan 8, 2025

View reviewed changes

modules/ROOT/pages/clustering/databases.adoc Show resolved Hide resolved

modules/ROOT/pages/clustering/databases.adoc Outdated Show resolved Hide resolved

modules/ROOT/pages/clustering/databases.adoc Outdated Show resolved Hide resolved

Apply suggestions from code review

b8f3f38

NataliaIvakina reviewed Jan 8, 2025

View reviewed changes

modules/ROOT/pages/clustering/databases.adoc Show resolved Hide resolved

Update modules/ROOT/pages/clustering/databases.adoc

ea870ec

nick-giles-neo reviewed Jan 8, 2025

View reviewed changes

NataliaIvakina added DO NOT MERGE and removed dev labels Jan 9, 2025

Merge branch 'dev' into 2024-12-seed-restore-until

8f3499d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add seed restore until section and remove 5.x seed provider info #2002

Add seed restore until section and remove 5.x seed provider info #2002

jackwaudby commented Dec 6, 2024

jackwaudby commented Dec 19, 2024

NataliaIvakina commented Dec 19, 2024

jackwaudby commented Dec 19, 2024 •

edited

Loading

NataliaIvakina Dec 23, 2024

jackwaudby Dec 23, 2024

nick-giles-neo Jan 6, 2025

nick-giles-neo Jan 7, 2025

NataliaIvakina Jan 8, 2025 •

edited

Loading

nick-giles-neo Jan 8, 2025

nick-giles-neo Jan 9, 2025

neo-technology-commit-status-publisher commented Jan 10, 2025


		This will seed the database with transactions up to, but not including transaction 123.

Add seed restore until section and remove 5.x seed provider info #2002

Are you sure you want to change the base?

Add seed restore until section and remove 5.x seed provider info #2002

Conversation

jackwaudby commented Dec 6, 2024

jackwaudby commented Dec 19, 2024

NataliaIvakina commented Dec 19, 2024

jackwaudby commented Dec 19, 2024 • edited Loading

NataliaIvakina Dec 23, 2024

Choose a reason for hiding this comment

jackwaudby Dec 23, 2024

Choose a reason for hiding this comment

nick-giles-neo Jan 6, 2025

Choose a reason for hiding this comment

nick-giles-neo Jan 7, 2025

Choose a reason for hiding this comment

NataliaIvakina Jan 8, 2025 • edited Loading

Choose a reason for hiding this comment

nick-giles-neo Jan 8, 2025

Choose a reason for hiding this comment

nick-giles-neo Jan 9, 2025

Choose a reason for hiding this comment

neo-technology-commit-status-publisher commented Jan 10, 2025

jackwaudby commented Dec 19, 2024 •

edited

Loading

NataliaIvakina Jan 8, 2025 •

edited

Loading