Init Code for Plugin (#1)

* wip: add updateRepo task

* wip: rename updateRepo to uploadDebian and add name+version param

renamed updateRepo to uploadDebian
-> added name and version parameter to override version and packagename
from debian file and for future clean task specification

* wip: refactor code and add removePackage Task

* wip: some renaming

* wip: add comments to PackageExtension

* wip: mulitpile fixes, see commit msg

added origin + label input param for Release File
added CleanPackages Step
  > added component and suite input param for that (has default
  stable/main value)
removed unused parameter from tasks
fixed Date Field
 -> CEST Date gives apt error
fixed poolbucketkey function:
  >the key must have component and not suite
added deleteObjects function to s3Client

* [Refactor] Add APT repository metadata configuration

This commit refactors the code to include additional APT repository metadata configuration options. The following changes were made:

In S3AptRepositoryPlugin.kt, the properties origin, label, suite, and component were added to the PluginExtension class and set to the corresponding values from the extension.
In RemovePackageTask.kt and UploadPackageTask.kt, the properties origin, label, suite, and component were added as inputs and assigned values based on the extension or the default values if not provided.
These additions allow users to customize the APT repository origin, label, suite, and component. The values can be set at the top level for the entire repository or overridden individually for specific Debian files or packages. The new properties are optional, allowing flexibility in the repository configuration.

Note: The origin, label, suite, and component properties have been introduced to provide enhanced customization options for the APT repository. Refer to the updated documentation for more details on how to use these properties effectively.

* wip: add tests

* wip: replaced hard coded dependencies

* wip: add readme and more unit tests

add more unit tests + mock tests
refactored a little of the main code for easier testing

* wip: add more unit tests

* wip: rename debianFiles and PluginExtension

renamed some input parameters and classes for better naming

* wip: rename integrationTest to integrationMain

* wip: refactor extensions

* wip: resolve conflicts from feat branch

* wip: clean some debian package tests

* wip: add more debianpackage tests

* refactor releaseinfo

refactored releaseinfo to stop being mutable

* wip: add parseToMap Test

* wip: remove mutable stuff

* refactor getFullBucketKey for better readability

* refactor: replace mockito with mockk

* wip: add some comments

* refactor: naming conventions

* wip: small improvments

* wip: add pre merge workflows

* wip: add publish plugin workflow for tags

* wip: typo readme

* change mutableList to list

* wip: rename architecture to packageArchitecture

in the next step i add release file field attributes and one of them are
architecture. for this i have to rename the architecture attribute from
the package to packageArchitecture.
This is also good, because all other package Attributes starting with
package... (packageName, packageVersion)

* wip: add release attributes/refactor to cleaner code
added release file attributes
 - added test for this
added comments
rename some stuff

---------

Co-authored-by: nvima <[email protected]>

.github/workflows/pre-merge.yaml

@@ -0,0 +1,25 @@
+name: Pre Merge Checks
+
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+    branches:
+      - '*'
+
+jobs:
+  gradle:
+    runs-on: ubuntu-latest
+    env:
+      GRADLE_PUBLISH_KEY: ${{ secrets.GRADLE_PUBLISH_KEY }}
+      GRADLE_PUBLISH_SECRET: ${{ secrets.GRADLE_PUBLISH_SECRET }}
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+      - name: Cache
+        uses: gradle/gradle-build-action@v2
+      - name: Validate
+        run: ./gradlew check validatePlugins --continue
+      - name: Integration Test
+        run: ./gradlew integrationTest --info

.github/workflows/publish-plugin.yaml

@@ -0,0 +1,24 @@
+name: Publish to Gradle Plugin Portal
+
+on:
+  push:
+    tags:
+      - '*'
+
+jobs:
+  gradle:
+    runs-on: ubuntu-latest
+    env:
+      GRADLE_PUBLISH_KEY: ${{ secrets.GRADLE_PUBLISH_KEY }}
+      GRADLE_PUBLISH_SECRET: ${{ secrets.GRADLE_PUBLISH_SECRET }}
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+      - name: Cache
+        uses: gradle/gradle-build-action@v2
+      - name: Validate
+        run: ./gradlew check validatePlugins --continue
+      - name: Integration Test
+        run: ./gradlew integrationTest
+      - name: Publish
+        run: ./gradlew publishPlugins

.gitignore

@@ -0,0 +1,44 @@
+.gradle
+build/
+!gradle/wrapper/gradle-wrapper.jar
+!**/src/main/**/build/
+!**/src/test/**/build/
+
+### IntelliJ IDEA ###
+.idea/modules.xml
+.idea/jarRepositories.xml
+.idea/compiler.xml
+.idea/libraries/
+*.iws
+*.iml
+*.ipr
+out/
+!**/src/main/**/out/
+!**/src/test/**/out/
+
+.idea/
+
+### Eclipse ###
+.apt_generated
+.classpath
+.factorypath
+.project
+.settings
+.springBeans
+.sts4-cache
+bin/
+!**/src/main/**/bin/
+!**/src/test/**/bin/
+
+### NetBeans ###
+/nbproject/private/
+/nbbuild/
+/dist/
+/nbdist/
+/.nb-gradle/
+
+### VS Code ###
+.vscode/
+
+### Mac OS ###
+.DS_Store

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 Liftric
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1 +1,138 @@
-TODO
+# Gradle S3 Apt Plugin 
+
+This Gradle plugin enables you to manage an Apt Repository in an Amazon S3 bucket. It's designed to assist with the task of handling Debian packages in an S3 environment. The plugin comes with three key tasks: `uploadPackage`, `removePackage`, and `cleanPackages`. 
+
+- The `uploadPackage` task allows for the upload of new Debian packages into the S3 Apt repository, including the necessary updating of the Release and Packages files. If no existing repository is found, a new one will be created.
+- The `removePackage` task is designed for the removal of specific versions of packages from the Packages file of a repository in an S3 bucket, with automatic updating of the Release file. The actual Debian file, however, will not be removed from the pool.
+- The `cleanPackages` task serves the purpose of deleting all Debian files that are no longer listed in the Packages file.
+
+> **Recommendation**: To create Debian packages from a Gradle project, there is this Gradle plugin https://github.com/nebula-plugins/gradle-ospackage-plugin
+
+> **Caution**: This plugin is still in the early stages of development and has not yet been extensively tested in a production environment. It is strongly recommended to backup your S3 bucket before using this plugin in a production setting to prevent any potential data loss.
+
+## Example
+
+```kotlin
+import com.liftric.apt.extensions.*
+
+plugins {
+    id("com.liftric.s3-apt-repository-plugin")
+}
+
+s3AptRepository {
+    bucket.set("s3-apt-repo")
+    region.set("eu-central-1")
+    accessKey.set(System.getenv("AWS_ACCESS_KEY"))
+    secretKey.set(System.getenv("AWS_SECRET_KEY"))
+    signingKeyPassphrase.set(System.getenv("PGP_PASSPHRASE"))
+    signingKeyRingFile.set(file("private.key"))
+    debPackage {
+        file.set(file("foobar_1.0.0-1_all.deb"))
+        packageArchitectures.set(setOf("all", "amd64"))
+    }
+}
+```
+
+### uploadPackage
+
+The `uploadPackage` task comes equipped with a range of both required and optional attributes.
+
+- **bucket**: *(Required)* The name of the S3 bucket where the Apt Repository resides.
+- **region**: *(Required)* The AWS region of the S3 bucket.
+- **accessKey**: *(Required)* The AWS Access Key for accessing the S3 bucket. Can be overridden in the `debPackage` section.
+- **secretKey**: *(Required)* The AWS Secret Key for accessing the S3 bucket. Can be overridden in the `debPackage` section.
+- **signingKeyRingFile**: *(Required)* The PGP private key file used for signing the Release files.
+- **signingKeyPassphrase**: *(Required)* The passphrase for the PGP private key.
+- **bucketPath**: *(Optional)* The path within the bucket to store the Apt Repository. If not specified, the repository is stored at the root of the bucket.
+- **endpoint**: *(Optional)* Custom S3 endpoint. Use this to override the default AWS S3 endpoint.
+- **override**: *(Optional)* Boolean value indicating whether to override existing version of a Package. By default, it is `true`.
+
+`Release File Attributes` can be override in s3AptRepository or per debPackage
+
+- **origin**: *(Optional)* The value of the 'Origin' field in the Release files. By default, it is 'Debian'.
+- **label**: *(Optional)* The value of the 'Label' field in the Release files. By default, it is 'Debian'.
+- **suite**: *(Optional)* The value of the 'Suite' field in the Release files. By default, it is 'stable'.
+- **component**: *(Optional)* The value of the 'Component' field in the Release files. By default, it is 'main'.
+- **architectures**: *(Optional)* The value of the 'Architectures' field in the Release files. By default, it is set correctly by the Plugin. If you need to override it, you can do so here.
+- **codename**: *(Optional)* The value of the 'Codename' field in the Release files. By default, it is not set.
+- **date**: *(Optional)* The value of the 'Date' field in the Release files. By default, it is set to the current date.
+- **releaseDescription**: *(Optional)* The value of the 'Description' field in the Release files. By default, it is not set.
+- **releaseVersion**: *(Optional)* The value of the 'Version' field in the Release files. By default, it is not set.
+- **validUntil**: *(Optional)* The value of the 'Valid-Until' field in the Release files. By default, it is not set.
+- **notAutomatic**: *(Optional)* The value of the 'NotAutomatic' field in the Release files. By default, it is not set.
+- **butAutomaticUpgrades**: *(Optional)* The value of the 'ButAutomaticUpgrades' field in the Release files. By default, it is not set.
+- **changelogs**: *(Optional)* The value of the 'Changelogs' field in the Release files. By default, it is not set.
+- **snapshots**: *(Optional)* The value of the 'Snapshots' field in the Release files. By default, it is not set.
+
+
+- **debPackage**: *(Required)* See below for more information.
+
+In the `debPackage` section, you can specify package-specific attributes. These can override the top-level attributes if needed:
+
+- **file**: *(Required)* The Debian package file to upload.
+- **packageArchitectures**: *(Required)* Set of architectures that the package supports.
+- **packageName**, **packageVersion**: *(Optional)* These attributes can be used to override the default package name and version extracted form the Debian File.
+- **origin**, **label**, **suite**, **component**, **architectures**, **codename**, **date**, **releaseDescription**, **releaseVersion**, **validUntil**, **notAutomatic**, **butAutomaticUpgrades**, **changelogs**, **snapshots** : *(Optional)* These attributes can be used to override the default Release file fields for the specific package.
+- **accessKey**, **secretKey**, **bucket**, **bucketPath**, **region**, **endpoint**: *(Optional)* These attributes can be used to override their respective top-level attributes for the specific package.
+
+
+
+
+
+### removePackage
+
+The `removePackage` task comes equipped with a range of both required and optional attributes.
+
+- **bucket**: *(Required)* The name of the S3 bucket where the Apt Repository resides.
+- **region**: *(Required)* The AWS region of the S3 bucket.
+- **accessKey**: *(Required)* The AWS Access Key for accessing the S3 bucket. Can be overridden in the `debPackage` section.
+- **secretKey**: *(Required)* The AWS Secret Key for accessing the S3 bucket. Can be overridden in the `debPackage` section.
+- **signingKeyRingFile**: *(Required)* The PGP private key file used for signing the Release files.
+- **signingKeyPassphrase**: *(Required)* The passphrase for the PGP private key.
+- **bucketPath**: *(Optional)* The path within the bucket to store the Apt Repository. If not specified, the repository is stored at the root of the bucket.
+- **endpoint**: *(Optional)* Custom S3 endpoint. Use this to override the default AWS S3 endpoint.
+- **origin**: *(Optional)* The value of the 'Origin' field in the Release files. By default, it is 'Debian'.
+
+`Release File Attributes` can be override in s3AptRepository or per debPackage
+
+- **label**: *(Optional)* The value of the 'Label' field in the Release files. By default, it is 'Debian'.
+- **suite**: *(Optional)* The value of the 'Suite' field in the Release files. By default, it is 'stable'.
+- **component**: *(Optional)* The value of the 'Component' field in the Release files. By default, it is 'main'.
+- **architectures**: *(Optional)* The value of the 'Architectures' field in the Release files. By default, it is set correctly by the Plugin. If you need to override it, you can do so here.
+- **codename**: *(Optional)* The value of the 'Codename' field in the Release files. By default, it is not set.
+- **date**: *(Optional)* The value of the 'Date' field in the Release files. By default, it is set to the current date.
+- **releaseDescription**: *(Optional)* The value of the 'Description' field in the Release files. By default, it is not set.
+- **releaseVersion**: *(Optional)* The value of the 'Version' field in the Release files. By default, it is not set.
+- **validUntil**: *(Optional)* The value of the 'Valid-Until' field in the Release files. By default, it is not set.
+- **notAutomatic**: *(Optional)* The value of the 'NotAutomatic' field in the Release files. By default, it is not set.
+- **butAutomaticUpgrades**: *(Optional)* The value of the 'ButAutomaticUpgrades' field in the Release files. By default, it is not set.
+- **changelogs**: *(Optional)* The value of the 'Changelogs' field in the Release files. By default, it is not set.
+- **snapshots**: *(Optional)* The value of the 'Snapshots' field in the Release files. By default, it is not set.
+
+- **debPackage**: *(Required)* See below for more information.
+
+In the `debPackage` section, you can specify package-specific attributes. These can override the top-level attributes if needed:
+
+- **file**: *(Required)* The Debian package file to extract Version and Package Name.
+- **packageArchitectures**: *(Required)* Set of architectures that the package supports.
+- **packageName**, **packageVersion**: *(Optional)* These attributes can be used to override the default package name and version extracted form the Debian File.
+- **origin**, **label**, **suite**, **component**, **architectures**, **codename**, **date**, **releaseDescription**, **releaseVersion**, **validUntil**, **notAutomatic**, **butAutomaticUpgrades**, **changelogs**, **snapshots** : *(Optional)* These attributes can be used to override the default Release file fields for the specific package.
+- **accessKey**, **secretKey**, **bucket**, **bucketPath**, **region**, **endpoint**: *(Optional)* These attributes can be used to override their respective top-level attributes for the specific package.
+
+
+### cleanPackages
+
+The `cleanPackages` task comes equipped with a range of both required and optional attributes.
+
+- **bucket**: *(Required)* The name of the S3 bucket where the Apt Repository resides.
+- **region**: *(Required)* The AWS region of the S3 bucket.
+- **accessKey**: *(Required)* The AWS Access Key for accessing the S3 bucket. Can be overridden in the `debPackage` section.
+- **secretKey**: *(Required)* The AWS Secret Key for accessing the S3 bucket. Can be overridden in the `debPackage` section.
+- **bucketPath**: *(Optional)* The path within the bucket to store the Apt Repository. If not specified, the repository is stored at the root of the bucket.
+- **endpoint**: *(Optional)* Custom S3 endpoint. Use this to override the default AWS S3 endpoint.
+- **suite**: *(Optional)* The value of the 'Suite' field in the Release files. By default, it is 'stable'.
+- **component**: *(Optional)* The value of the 'Component' field in the Release files. By default, it is 'main'.
+
+## License
+
+This S3 Apt Repository Plugin is released under MIT License.

build.gradle.kts

@@ -0,0 +1,90 @@
+@Suppress("DSL_SCOPE_VIOLATION") // IntelliJ incorrectly marks libs as not callable
+plugins {
+    `maven-publish`
+    alias(libs.plugins.kotlinJvm)
+    alias(libs.plugins.kotlinSerialization)
+    alias(libs.plugins.dockerCompose)
+    alias(libs.plugins.gradlePluginPublish)
+    alias(libs.plugins.nemerosaVersioning)
+}
+
+group = "com.liftric"
+version = with(versioning.info) {
+    if (branch == "HEAD" && dirty.not()) {
+        tag
+    } else {
+        full
+    }
+}
+
+repositories {
+    mavenCentral()
+    gradlePluginPortal()
+}
+
+sourceSets {
+    val main by getting
+    val integrationMain by creating {
+        compileClasspath += main.output
+        runtimeClasspath += main.output
+    }
+}
+
+tasks {
+    val test by existing
+    withType<Test> {
+        useJUnitPlatform()
+        testLogging {
+            events("passed", "skipped", "failed")
+        }
+        systemProperty("org.gradle.testkit.dir", gradle.gradleUserHomeDir)
+    }
+
+    register<Test>("integrationTest") {
+        val integrationMain by sourceSets
+        description = "Runs the integration tests"
+        group = "verification"
+        testClassesDirs = integrationMain.output.classesDirs
+        classpath = integrationMain.runtimeClasspath
+        mustRunAfter(test)
+        useJUnitPlatform()
+    }
+}
+
+
+gradlePlugin {
+    val integrationMain by sourceSets
+    testSourceSets(integrationMain)
+    plugins {
+        create("s3-apt-repository-plugin") {
+            id = "$group.s3-apt-repository-plugin"
+            implementationClass = "$group.apt.S3AptRepositoryPlugin"
+            displayName = "s3-apt-repository-plugin"
+        }
+    }
+}
+
+pluginBundle {
+    website = "https://github.com/Liftric/s3-apt-repository-plugin"
+    vcsUrl = "https://github.com/Liftric/s3-apt-repository-plugin"
+    description = "A Gradle Plugin for managing an APT Repository on S3"
+    tags = listOf("s3", "apt", "repository", "plugin", "gradle", "debian")
+}
+
+dependencies {
+    implementation(libs.kotlinStdlibJdk8)
+    implementation(libs.apacheCommons)
+    implementation(libs.xz)
+    implementation(libs.bouncyCastleGPG)
+    implementation(libs.bouncyCastleProvider)
+    implementation(libs.awsS3)
+
+    testImplementation(libs.junitJupiter)
+    testImplementation(libs.mockk)
+
+    "integrationMainImplementation"(gradleTestKit())
+    "integrationMainImplementation"(libs.junitJupiter)
+    "integrationMainImplementation"(libs.testContainersJUnit5)
+    "integrationMainImplementation"(libs.testContainersMain)
+    "integrationMainImplementation"(libs.minio)
+}

gradle.properties

@@ -0,0 +1 @@
+kotlin.code.style=official

gradle/wrapper/gradle-wrapper.properties

@@ -0,0 +1,5 @@
+distributionBase=GRADLE_USER_HOME
+distributionPath=wrapper/dists
+distributionUrl=https\://services.gradle.org/distributions/gradle-7.4.2-bin.zip
+zipStoreBase=GRADLE_USER_HOME
+zipStorePath=wrapper/dists