Swift Package Manager Support

Expected Behavior

Being able to install SQLCipher using a Swift Package Manager package maintained by Zetetic

Actual Behavior

In order to use SQLCipher with Swift Package Manager, a user must generate an amalgamation themselves and create their own Package.swift, e.g. https://github.com/antwork/SQLCipher or groue/GRDB.swift#556

Steps to Reproduce

N/A

SQLCipher version: 4.4.0

Hello,

While I wish this was a pull request adding a Package.swift to the repo instead of an issue requesting one, SQLCipher's amalgamation step complicates this. As far as I know there is currently no SPM equivalent to the prepare_command used in SQLCipher.podspec.json to generate the amalgamation. An SPM package can easily be made by including an amalgamation as seen in the links above, but since the amalgamation is quite large, it understandably is not included in the SQLCipher repo.

Would Zetetic be open to maintaining a repo that's an iOS/macOS-compatible amalgamation and Package.swift, similar to the one linked above? Since installing dependencies with Swift Package Manager is now directly integrated into Xcode, it will inevitably replace CocoaPods as the de-facto dependency management solution for Xcode projects.

Thank you for this project and taking the time to read this!

@jzzocc

Hello @jzzocc - thanks for bringing this up. Let us take a look a closer look at this and we'll give some consideration to the approach.

It looks there is an unofficial and just started (not finished) work here:

https://github.com/antwork/SQLCipher/blob/master/Package.swift

This Package.swift could serve as a base maybe...

Any progress/thoughts on this? Trying hard to avoid integrating CocoaPods in our project.

@sjlombardo

@sjlombardo any updates on that?

Unfortunately I don't have any good news here. We looked into this in detail, but the lack of any pre-processing mechanism in Swift Package manager is a deal breaker. We do not really want to maintain and manage a separate amalgamated copy of SQLCipher and the related requisite headers separately from the official sources.

My suggestion at this point would be that anyone interested in supporting this should go and log a request with Swift Package manager to support a pre-preprocessing mechanism. Perhaps this might be a good thread to comment on, as it proposes an extension mechanism, and a prebuild facility:

https://forums.swift.org/t/pitch-swiftpm-extensible-build-tools/44715/5

If enough people support this sort of feature it might be added to Swift Package manager in the future. This sort of thing is really a critical feature for a package manager to leave out, since it rules out using it with many more complex projects that include code-preprocessing, assembly, etc.

We will leave this issue open for now. If Swift Package manager adds this important feature in the future we'd be happy to reconsider hosting a Package.swift.

@sjlombardo

@sjlombardo actually community liked it a lot and it passed already reviews and revisions and is now accepted state. Can you double check that prebuild tools provided in accepted proposal https://github.com/apple/swift-evolution/blob/main/proposals/0303-swiftpm-extensible-build-tools.md will be enough to make SQLCipher to SPM. Probably will take a year till it will be released, but its a good news it got accepted.

Who is interested in this topic, then SPM extensible build tools was implemented in Swift 5.6 version.
It will probably come out in March/April as Swift release cycle is in every half year and 5.5 comes out this month.
https://github.com/apple/swift-evolution/blob/main/proposals/0303-swiftpm-extensible-build-tools.md

@kristohear

@kristohear thank you for the update on this! We'll keep an eye out for the 5.6 version and will plan appropriate changes!

Just a little bump and to note that Swift 5.6 was released today with support for extensible build tools included!

https://www.swift.org/blog/swift-5.6-released/

Thank you for bringing this to our attention, we'll take a look soon!

This is a work in progress, but we've stalled out a bit. While we were able to generate a package and corresponding buildTool plugin, the API appears to be very limited with respect to calling arbitrary build scripts. It doesn't seem possible to directly execute the ./configure and make sqlite3.c commands necessary to generate the amalgamation from source, and neither tool can be declared as binaryTargets. It seems like some additional executable swift wrapper may be necessary. In addition, there seems to be some interference from the sandbox that the plugins execute in.

If anyone has experience with directly executing pre-existing (non-swift) build tools from the plugin API, we'd love to hear about it.

@sjlombardo

Hello @sjlombardo, thanks for having explored the subject!

Did Zetetic consider contacting Apple and the SPM team in order to report the needs of your (many) users? The needed features would be huge SPM improvements. If a quickly brush them out, this would be:

Make SPM able to build SQLCipher as a binary target.

Use case: I can add a dependency on SQLCipher in my SPM package.
Make SPM able to build SQLite as a binary target.

Use case: I can add a dependency on SQLite in my SPM package.
Make SPM able to support all build options of SQLCipher and SQLite.

Use case: When I add a SPM dependency on SQLCipher or SQLite, I can control the compile-time options and extensions supported by both libraries.
Make SPM able to support all build options of SQLCipher and SQLite in a dependency.

Use case: When I add a SPM dependency that itself depends on SQLCipher or SQLite, I can control the compile-time options and extensions supported by both libraries, AND tell the dependency about enabled SQLCipher/SQLLite features (think GRDB.swift, SQLite.swift, and all Swift wrappers around SQLite or SQLCipher)

I'm really not sure of the awareness of the SPM team about these needs.

@groue

We've pushed out a work in progress branch for switfpm. It includes a plugin and an executableTarget that are able to build the amalgamation. It works within the sandbox limitations by copying the requisite source files into the sandbox-permitted plugin working directory prior to creating the amalgamation.

The main limitation here is that, while the package is able to generate the files necessary, SwiftPM Plugins don't appear to support header files as outputs (it just generates a warning about unknown file type for the .h). This means the generated sqlite3.h is not available to the build system after it is created. To use the package in an application you must add a pre-compile run script (or have some other means) to copy the generated header file to a location in the header search path. This is an example Run Script that does so:

# locate the base folder for swift package manager source packages
SOURCE_PACKAGES="${BUILD_DIR%Build/*}SourcePackages"

# headers will be copied to the build products include directory 
# for the target where they will be available as part of the default
# header search path
mkdir -p "$BUILT_PRODUCTS_DIR/include"

# locate the output folder where the generated headers were placed
# by the AmalgamationPlugin. SwiftPM names the temporary directory
# the same as the plugin name, so there will be multiple folders
# named AmalgamationPlugin under SourcePackages. The correct location,
# the sandbox-permitted plugin working folder will have a "gen" directory
# immediately under it where the header files are located
for ap in `find "$SOURCE_PACKAGES" -type d -name "AmalgamationPlugin"`; do
    if [ -d "$ap/gen" ]; then
        for file in `find "$ap/gen" -name "*.h"`; do 
            echo "Copying $file to $BUILT_PRODUCTS_DIR/include"
            cp "$file" "$BUILT_PRODUCTS_DIR/include"
        done
        break;
    fi
done

Of course the requirement to use a Run script is a hack that eliminates a lot of the "plug and play" benefit, so this is unlikely to be merged up or released as is. That said, If anyone wants to give this a try or has ideas on how to make plugin-generated headers available to the building application, I'm open to feedback.

@groue - Unfortunately we don't have any contacts at Apple and wouldn't even know where to start. If you know the right people and are willing to make an introduction we'd be happy to support the process.

For tracking I've posted a question about this on the Swift forum.

You did it better than I would have done that myself :-)

@groue

If folks really want this to happen, then please listen to the suggestion from @groue and make your voices heard on the swift forum, feedback assistant, etc. This is the underlying issue:

https://forums.swift.org/t/plugin-generated-header-files-are-not-available-from-project/56875/5

We (the SQLCipher team) already coded the entire SPM plugin integration over 2 years ago. However, without the ability to expose generated headers to the application it will not work. This is a huge deficiency: plugin-generated c files without corresponding headers are not at all useful. This is a broken SPM feature.

So far we've had 20+ post on this thread about the issue with SQLCipher, and ONE about the underlying swift issue. Please go to the forum issue, drum up support, and advocate for a fix in the proper place.

https://blog.cocoapods.org/CocoaPods-Support-Plans/

Does the swift package manager support swift file export? Then is it possible to generate a swift file that is in some sense a "bridge" between the caller and SQLCipher.

Swift 5.9 supports calling C++ code directly without Objective-C.

Here are some irrelevant contents:

Details

Even though I "re-activated" the post on the Swift forum, I still couldn't get the attention of the Swift team. As someone who is not familiar with the Swift forum, I don't know how to "re-update" this matter or get enough attention from the Swift team except replying in this post.

So I can only rack my brains to think about how to make SQLCipher support SPM on the existing basis.

CocoaPods has announced that it will enter the maintenance period, but SQLCipher does not support SPM yet. I think this is a major security risk for iOS applications. After all, many people are reluctant to use CoreData or SwiftData, and people love SQLite.

@marcprux

We've added pure SwiftPM integration of SQLCipher into our project by adding a separate SQLExt C module that includes the sqlite amalgamation as well as the LibTomCrypt libraries. You can see the package definition at https://github.com/skiptools/skip-sql/blob/main/Package.swift and the SQLExt module at https://github.com/skiptools/skip-sql/tree/main/Sources/SQLExt.

Others could use the same technique, or just use our SkipSQLPlus package, which includes sqlcipher as well as FTS5 and other extensions and has the benefit of running on both Darwin platforms (iOS and macOS) as well as Android. Docs are at https://skip.tools/docs/modules/skip-sql/#encryption

@marcprux Could you shed some light on how you generated the amalgamation? Say I were to clone this repository, which command would one run to get to the sqlite3.h and sqlite3.c file you are referencing? I was fiddling with this a bit, but for some reason I have to always add the SQLITE_DEBUG flag, else the project won't compile

Could you shed some light on how you generated the amalgamation?

Take a look at https://github.com/skiptools/swift-sqlcipher, which is a more recent iteration on the same idea, but without any external dependencies, and with native support for macOS, iOS, Windows, Linux, and Android.

The amalgamation is created/updated with the build_sqlcipher.sh script, which pulls in the latest version (currently 4.6.1) of sqlcipher's patched sqlite amalgamation. It is a pure source build that bundles libtomcrypt as the crypto provider.

SQLCipher & the Swift Package Manager | Livefront Talks 2023

We can use this xcframework iOS library for now.
https://github.com/chanonly123/SQLCipher-SPM.git

Hello
Do we have an official fix for SPM that we can expect ?
podfile will be deprecated in 5 months now, and alternative do not seem to work

There's nothing on the horizon (Swift 6.2 and beyond) that will facilitate the sort of Swift Package Manager pre-processing capabilities that would be needed for the ideal solution. So I think that this sentiment from 2021 ought to be reconsidered:

We do not really want to maintain and manage a separate amalgamated copy of SQLCipher and the related requisite headers separately from the official sources.

The solution we've been using in swift-sqlcipher is simple to maintain: every time there is a new sqlcipher release, we just run a script to update and tag a derived repository with the sqlite + sqlcipher + libtomcrypt sources, and then SwiftPM can just include it as a source dependency.

The Swift 6.1 addition of package traits makes this even more powerful, since the various (opinionated) preprocessor settings in Package.swift could instead be controlled by published traits, which would allow the dependent project a great deal of control over the SQLite options when building from source (see the CSQLite project for an example of this in action).

I would be happy to work with the sqlcipher team on putting something together. It might even help push forward my GRDB PR (groue/GRDB.swift#1708) to add SQLCipher support via SwiftPM.

Yes, the SQLCipher team is actively working on a new SPM approach for an upcoming SQLCipher release. We'll post more information soon.

I'd like to start off by thanking everyone who contributed to this thread, or the companion thread on the Swift Forums, for your patience and feedback. I'm pleased to say that starting with SQLCipher 4.10.0, we are now officially supporting SPM with the SQLCipher.swift package.

This package is designed specifically so that it can be used either directly by applications, or as a dependency by other Swift packages that would like to expose SQLCipher encryption.

The package provides the following benefits:

It will be the "official" Swift Package for SQLCipher, maintained directly by the SQLCipher development team. This avoids having to trust third-party sources for SQLCipher packages.
The SQLCipher team guarantees that it will be updated immediately (e.g. same day) with every single release of SQLCipher. This eliminates the potential for outdated security components.
The library will work well with other Community Edition platforms (e.g. Android and our previous CocoaPods implementation) that are maintained by the SQLCipher core team. It's built with a consistent set of SQLCipher/SQLite features so there won't be any guesswork of what features to enable or disable for each build.
It contains binary targets supporting the full line of Apple SDKs. This provides compact and efficient integration, without the need to copy, clone, compile, or maintain large and redundant amalgamated C files. This also means that the amalgamation steps, custom SPM plugins, and other complexities mentioned earlier in this thread have been eliminated.
The package dependency can be easily overridden using a local Swift Package of the same name. That means that if a library depends on SQLCipher.swift, but a particular user needs to do something special (e.g. custom build), they can simply provide their own local SQLCipher.swift package and use it in place. This also works for SQLCipher Commercial and Enterprise packages.
It will be the only package to receive public support through the SQLCipher forums and GitHub issues. We did publish the 4.10.0 CocoaPod for this release, but we will no longer be updating it for future releases.

Migration from existing third-party Swift packages or CocoaPods to SQLCipher.swift should be very easy. This package uses SQLCipher 4.10.0, so modern databases will continue to work without changes, and there should not be any major breaking changes (e.g. those recently introduced when compiling from source due to major changes to the SQLite build system or SQLCipher initialization). Even if an application is using a very old version of SQLCipher, i.e. version 3 or lower, then they can still use this library with compatibility settings. We have also published complete integration instructions for both Application use and Library integrations.

We hope that this new package will be adopted by library maintainers and applications going forward. I will leave this GitHub issue open for a short while to answer any questions that might crop up.

I'm sorry to say, but I find this solution disappointing. Binary .xcframework dependencies are only supported for Apple platforms (macOS, iOS, etc), so this won't work for Linux, Android, Windows, BSD, etc. Additionally, you are dumping the frameworks (totaling 33M) directly into the git repository, and since git doesn't tend to handle binary diffs very well, that will inflate to 66M for the next release, 99M for the following, and so on, all of which will need to be downloaded by SwiftPM every time this dependency is used.

What was it about distributing SQLCipher as a source amalgamation that you found cumbersome? It has been trivial to maintain for swift-sqlcipher, which I've been updating within a day or so of the official releases (I just released 4.10.0 a few hours ago).

@marcprux

Hello @marcprux - thanks for getting back to us with feedback on this. You make some excellent points which I'll address here.

First, I wanted to mention that this is our initial release with support for SPM. We will almost certainly make modifications from here based on feedback and usage. So, please consider this an initial step in an ongoing process, not a final destination.

With that in mind, our initial goal is to support the Apple ecosystem where we have an immediate need to replace CocoaPods and know firsthand that we have a large number of developers using Swift. That is not to say that we'll never support other platforms, just that Apple is our focus now. We will continue to evaluate and adjust course in the future as necessary.

We really appreciate you pointing out the shortcoming of the way SwiftPM full clones git repos. This will definitely be addressed with the next release. Instead of placing the binary target in the repo itself, we will publish a compressed binary target zip attached to a proper GitHub Release. This appears to be a fairly common approach for other Swift Packages and will cleanly mitigate the risk of long term repository growth.

I'd also like to clarify the comment in my earlier post regarding "outdated security components". That was not intended as a shot at your swift-sqlcipher project (or any other specific project), and I apologize if it came across that way. It looks like you've done a great job of keeping up with upstream SQLCipher releases. However, there are dozens of other third-party SQLCipher libraries that have not been as diligent as you. Many projects unfortunately become outdated by multiple versions or many years. Any application adding a dependency needs to make their own decisions about this, but it's clearly an important factor. The earlier post simply meant to call out that one significant benefit of using this package is that the SQLCipher developers guarantee to update it as part of the standard release process.

Lastly, you made a comment/question about the amalgamation which warrants a response. We do not like the idea of distributing an amalgamation separate from the official canonical sources of the SQLCipher project. It creates an implied contract that the amalgamation is directly supported, which it is not. It duplicates the project history, but in a convoluted and hard-to-track way. It's a slippery slope that could lead to projects depending on what becomes perceived as an "official" amalgamation instead of the full sources. It could also increase compatibility issues with different features / flags. All of these constitute a support or maintenance burden we feel is undesirable.

Not everyone agrees with this position, however, it may be shared by the upstream SQLite development team. The SQLite team has recently a) completely revamped the build process to make use of the full source tree the standard b) referred to canonical source as the primarily supported package c) downgraded the position of amalgamation for downloads, and d) mentioned that amalgamation downloads may be removed at some time in the future.

As a result of these factors an amalgamated distribution is not our preferred approach. As evidenced by this thread, we undertook a good faith effort over an extended period of time to build a package that would work from the official sources. We attempted to lobby the SPM developers to fix the technical issues that prevented it from working. Unfortunately it just doesn't seem like there is serious interest in SPM fully supporting C as a language (i.e. specifically one that can use headers in generated sources). Thus, at least for the time being, it seems like the binary target is the appropriate middle ground (though we will reevaluate going forward).

Thanks again for the thoughtful feedback, we will definitely take it into account as we iterate on our support for SPM.

Instead of placing the binary target in the repo itself, we will publish a compressed binary target zip attached to a proper GitHub Release

Have you considered Git LFS? Uploading zip files in releases is fine, but it would be devastating for our "automatically syncing GitHub repositories with our internal GitLab" scenario... We'd need to write a lot of extra code to call the GitLab API to sync the files included in the release, and it might not even be possible.

In addition, if the xcframework package size problem is solved by uploading the zip in the RELEASE, then the SQLCipher.swift repository is no longer necessary. The xcframework file can be distributed in the RELEASE of the main repository, and only a Package.swift file needs to be added to the main repository.

It is good to hear that you are open to reevaluating the approach, and I do sympathize with the immediate need to provide a drop-in replacement for the CocoaPods distribution. I also share your frustration with SwiftPM's limitations around C-based modules.

I'll enumerate here my points in favor of a source distribution so that they might be taken into consideration for future direction(s):

Debuggable: With a source distribution, you get NDEBUG/SQLITE_DEBUG for free with Debug/Release configurations, plus crashes can jump directly to the source line in Xcode, and breakpoints can be set/removed for fine-grained analysis.
Configurable: When combined with SwiftPM package traits (see above), all of SQLite's features and performance/debugging flags could be toggled on/off and customized by the consuming package.
Auditable/Reproducible: As package security and bills-of-materials become more important in the current threat landscape, being able to audit and reproduce a build is very valuable. Apps like Signal utilize this for trust, and certain app stores mandate reproducibility as a prerequisite for app distribution, which necessarily disqualifies apps that have opaque binary-only dependencies.
Performant: The amalgamation distribution is inherently faster: "Combining all the code for SQLite into one big file makes SQLite easier to deploy — there is just one file to keep track of. And because all code is in a single translation unit, compilers can do better inter-procedure and inlining optimization resulting in machine code that is between 5% and 10% faster."
Portable: It bears repeating that the distribution of such an important package in a Darwin-only format hurts the broader cross-platform Swift effort. I do not speak for the Swift project as a whole, but as a founder of the Swift Android Workgroup, we are invested in expanding the ecosystem of Swift packages that can be used across the entire range of supported platforms.

I suppose I can see some concern about the duplication of project history, but I feel that a bit explanatory documentation (like "These are derived sources; for the canonical sources, go [here] instead…") would go a long way towards mitigating any speculated support or maintenance burdens. But regarding the comment about upstream sharing these concerns:

The SQLite team has recently … c) downgraded the position of amalgamation for downloads, and d) mentioned that amalgamation downloads may be removed at some time in the future.

Do you have a reference for this? The amalgamation is listed prominently as the second artifact on the downloads page, and while I am certain the sqlcipher team is much better tuned into the active discussions of the upstream sqlite project, I can't find any cautionary text about it being deprecated in the future, either on their official The Amalgamation Versus Canonical Sources page, or in any of the mailing lists or discussion forums. This note in the Advantages Of Building From The Amalgamation section makes me think that the amalgamation is not going away any time soon:

You Will End Up Using The Amalgamation Regardless

If you build from canonical sources, the Makefile first builds the amalgamation and then compiles the amalgamation into binary machine code. Why not just skip that first step and go right to compiling the amalgamation directly?

I hope you will find some of these points convincing. I would be happy to "donate" the swift-sqlcipher package as a swift-sqlcipher-experimental repository under the sqlcipher org and help iterate on it (with a prominent disclaimer that it is unsupported) if you are interested in exploring this option without committing to it for the long term.

Navigation Menu

Repository navigation

Swift Package Manager Support #371

Expected Behavior

Actual Behavior

Steps to Reproduce

26 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Swift Package Manager Support #371

Description

Expected Behavior

Actual Behavior

Steps to Reproduce

Activity

sjlombardo commented on Sep 4, 2020

nacho4d commented on Nov 2, 2020

eirikvaa commented on Jan 11, 2021

kristohear commented on Apr 26, 2021

sjlombardo commented on Apr 27, 2021

kristohear commented on Apr 29, 2021

kristohear commented on Sep 6, 2021

sjlombardo commented on Sep 9, 2021

mikecsh commented on Mar 15, 2022

sjlombardo commented on Mar 21, 2022

sjlombardo commented on Apr 18, 2022

groue commented on Apr 19, 2022

sjlombardo commented on Apr 20, 2022

sjlombardo commented on Apr 21, 2022

groue commented on Apr 22, 2022

26 remaining items

sjlombardo commented on Jul 24, 2024

b95505017 commented on Aug 15, 2024

rakuyoMo commented on Aug 16, 2024

Craz1k0ek commented on Dec 13, 2024

marcprux commented on Dec 13, 2024

qsd-faris commented on Mar 6, 2025

chanonly123 commented on May 10, 2025

altagir commented on Jul 9, 2025

marcprux commented on Jul 9, 2025

sjlombardo commented on Jul 10, 2025

sjlombardo commented on Aug 5, 2025

marcprux commented on Aug 5, 2025

sjlombardo commented on Aug 7, 2025

rakuyoMo commented on Aug 8, 2025

marcprux commented on Aug 9, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions