How to Implement Inter-Process Database Communication Rid of ContentProvider?
How to implement inter-process database communication rid of ContentProvider?
Most of us Android developers will never have an encounter with such a requirement like database IPC, because the Android framework has already provided an approach called
ContentProvider. But do you know how it works and what if your server process is not even a standard Android application? That’s what we are going to talk about.
Begin with ContentProvider
ContentProvider gives Android the capability of providing a variety of content between applications in a encapsulated and controlled way. This mechanism is so widely used from reading contacts from the phonebook to picking an image from the gallery. As a developer, if you’ve ever used a ContentProvider, you must have noticed that the ContentProvider framework provides a set of APIs, which are similar to the structural query language, although this does not mean that the provider must store its data in a relational database. You can refer to this page to learn basics about ContentProvider.
There are many types of data but in conclusion, two types: structured data and binary file data. For files, the data is queried by ContentResolver.open*() , where the caller can retrieve the stream or the file descriptor, so its IPC mechanism is FileDescriptor exposing. As for structural data, in most cases, they are stored in a relational database like SQLite database. They can be queried by ContentResolver.query(), which returns a Cursor just like querying a local database. It’s obvious that the cursor is the key of database IPC. Now what we want to know is how to transact a cursor inter-process.
How to transact a Cursor inter-process?
What will you do if you are the designer of ContentProvider?
Concepts
To help your further reading, we need to clarify two nouns.
The caller process
The process who requires content.
The callee process
The process who provides content.
# The quite straight-forward idea
The class Cursor is just an interface defining a cursor’s behaviors and we don’t need to care about what is exactly running underneath. Upholding this concept, we may come up with a straight-forward idea, that is AIDL. We can simply define an AIDL mimicking the Cursor interface:
Note: To abridge the article, we will just pick several interfaces for the illustration:
// IRemoteCursor.aidl
interface IRemoteCursor {
int getCount();
boolean moveToNext();
// ...and so on
void close();
}
After a successful build, we can define a class called RemoteCursor inheriting from Cursor . It delegates the remote calls in the caller process.
// RemoteCursor.kt
private class RemoteCursor(private val binder:IRemoteCursor): Cursor{
override fun getCount(): Int {
return binder.count
}
override fun moveToNext(): Boolean {
return binder.moveToNext()
}
// ...and so on
override fun close() {
binder.close()
}
}
/**
* Reconstruct the cursor.
*/
fun IBinder.asCursor(): Cursor {
return RemoteCursor(RemoteCursor.Stub.asInterface(this))
}
And also we need to do real work in the callee process.
// RemoteCursorImpl.kt
private class RemoteCursorImpl(private val cursor:Cursor): IRemoteCursor.Stub() {
override fun getCount(): Int {
return cursor.count
}
override fun moveToNext(): Boolean {
return cursor.moveToNext()
}
// ...and so on
override fun close() {
cursor.close()
}
}
/**
* Deconstruct the cursor.
*/
fun Cursor.asBinder(): IRemoteCursor {
return RemoteCursorImpl(this)
}
Everything is done now. If we want to do a remote database query, we can just build a RemoteCursorImpl from a normal cursor in the callee process and transact it as a binder to the caller process, where we can reconstruct the cursor as RemoteCursor from the binder.
The hidden performance issue
Everything seems just perfect and working as expected, but there is a hidden performance issue. If the table we were querying contains a large number of records and you would like to read it all, the performance issue would be very significant, because there were too many IPC calls and too many memory copies.
The following code snippet is a common cursor iteration boilerplate:
database.query("SELECT projections FROM table_name").use {
while (it.moveToNext()) {
val str_field = it.getString(0)
val int_field = it.getInt(1)
// and more
}
}
You can see that whenever we iterate a cursor, the application needs to do approximately as many as row*col times of IPC calls and memory copies. Even though the IPC call and memcpy are fast, but due to the time complexity of a cursor iteration, the more rows and columns you want to query, the more unbearable the performance issue will be, which could finally lead to a performance disaster.
# How does the Android framework do it?
Let’s dive into the source code. In ContentProviderNative, where the actual ContentProvider is implemented, we will see how they reconstruct the cursor from the callee process:
// ContentProviderNative.java
@Override
public Cursor query(@NonNull AttributionSource attributionSource, Uri url,
@Nullable String[] projection, @Nullable Bundle queryArgs,
@Nullable ICancellationSignal cancellationSignal)
throws RemoteException {
BulkCursorToCursorAdaptor adaptor = new BulkCursorToCursorAdaptor();
Parcel data = Parcel.obtain();
Parcel reply = Parcel.obtain();
try {
// ...
mRemote.transact(IContentProvider.QUERY_TRANSACTION, data, reply, 0);
// ...
if (reply.readInt() != 0) {
BulkCursorDescriptor d = BulkCursorDescriptor.CREATOR.createFromParcel(reply);
Binder.copyAllowBlocking(mRemote, (d.cursor != null) ? d.cursor.asBinder() : null);
adaptor.initialize(d);
} else {
adaptor.close();
adaptor = null;
}
return adaptor;
} catch (RemoteException ex) {
adaptor.close();
throw ex;
} catch (RuntimeException ex) {
adaptor.close();
throw ex;
} finally {
data.recycle();
reply.recycle();
}
}
The final cursor returned is an instance of class BulkCursorToCursorAdaptor, which follows the adaptor design pattern. The BulkCursorToCursorAdaptor is initialized from a BulkCursorDescriptor, which is parcelable. So in the callee side, there must be something that converts a cursor into a BulkCursorDescriptor and transact it to the caller side and the caller side reconstructs the cursor with BulkCursorToCursorAdaptor. The idea seems just like what we’ve thought before, but will this implementation fall into a performance issue? Let’s take a look at what BulkCursorToCursorAdaptor.initialize() does:
// BulkCursorToCursorAdaptor.java
public void initialize(BulkCursorDescriptor d) {
mBulkCursor = d.cursor;
mColumns = d.columnNames;
mWantsAllOnMoveCalls = d.wantsAllOnMoveCalls;
mCount = d.count;
if (d.window != null) {
setWindow(d.window);
}
}
This method simply assigns fields from BulkCursorDescriptor to it’s member fields, but there is something new called mBulkCursor(an instance of IBulkCursor). What is it? In here, we find that the IBulkCursor is an interface defining something similar to a cursor but much simpler. The most remarkable method is getWindow(), which returns a CursorWindow, while the CursorWindow is parcelable. Then everything seems to be clear. The essence of a cursor is its CursorWindow. We can reconstruct a cursor from a cursor window and some metadata. Now let’s find out how getWindow() is implemented. There is another adaptor called CursorToBulkCursorAdaptor, which yields a BulkCursorDescriptor from a normal cursor and is the real implementation of IBulkCursor:
// CursorToBulkCursorAdaptor.java
@Override
public CursorWindow getWindow(int position) {
synchronized (mLock) {
throwIfCursorIsClosed();
if (!mCursor.moveToPosition(position)) {
closeFilledWindowLocked();
return null;
}
CursorWindow window = mCursor.getWindow();
if (window != null) {
closeFilledWindowLocked();
} else {
window = mFilledWindow;
if (window == null) {
mFilledWindow = new CursorWindow(mProviderName);
window = mFilledWindow;
} else if (position < window.getStartPosition()
|| position >= window.getStartPosition() + window.getNumRows()) {
window.clear();
}
mCursor.fillWindow(position, window);
}
if (window != null) {
// Acquire a reference to the window because its reference count will be
// decremented when it is returned as part of the binder call reply parcel.
window.acquireReference();
}
return window;
}
}
We can see that the cursor window is lazily filled via mCursor.fillWindow(position, window) as needed. You might wonder what is a cursor window? As the documentation explained, a cursor window is
A buffer containing multiple cursor rows.
So once a cursor window is filled, the caller does not need to invoke IPC calls and do memory copies for each column and each row, which largely cuts down the time consumption and consequently gets freed from the performance issue.
Get rid of ContentProvider
Now that we’ve known the principle of cursor IPC, we can work on the database IPC without ContentProvider. We just need to mimics the Android framework. The following code snippets use SQLite database as an example.
Be careful of hidden APIs!
You might think that we can just use those APIs like IBulkCursor in the Android framework by reflection or linking. Yes, this may work but do rememeber that those are hidden apis and are not designed to be used by third-party apps, which are expected to be run on various devices. Even if you could bypass the hidden api restrictions above Android P, their implementations may vary on different Android releases or even on different ROMs. You’d better write the codes in your project.
RemoteSQLiteDatabase
Firstly, we need to abstract a remote sqlite database representing a sqlite database in the callee process to be used in caller process. Thanks to androidx.sqlite, they provide a fine abstraction of a sqlite database called SupportSQLiteDatabase. Now we just need to copy its interfaces into an AIDL file. There are a lot of interfaces, you can just count those necessary in. In our illustration, we will only show several interfaces.
// IRemoteSQLiteDatabase.aidl
interface IRemoteSQLiteDatabase {
void beginTransaction();
void endTransaction();
BulkCursorDescriptor query(String query,in String[] bindArgs);
// ...and so on
void close();
}
You might have noticed that we return a BulkCursorDecriptor in query method, because the BulkCursorDecriptor is parcelable and contains all we need to rebuild a cursor. Don’t forget to import it.
// BulkCursorDescriptor.java
public final class BulkCursorDescriptor implements Parcelable {
public IBulkCursor cursor;
public String[] columnNames;
public boolean wantsAllOnMoveCalls;
public int count;
public CursorWindow window;
}
Now what we need is IBulkCursor, we can define it in AIDL as well.
// IBulkCursor.aidl
interface IBulkCursor {
/**
* Gets a cursor window that contains the specified position.
* The window will contain a range of rows around the specified position.
*/
CursorWindow getWindow(int position);
/**
* Notifies the cursor that the position has changed.
* Only called when {@link #getWantsAllOnMoveCalls()} returns true.
*
* @param position The new position
*/
void onMove(int position);
void close();
}
And there are CursorToBulkCursorAdaptor and BulkCursorToCursorAdaptor. I will no longer paste them here because I don’t want to make the article too long. They can be simply copied from the Android framework source codes with some small modifications. Now let’s combine these all.
// RemoteSQLiteDatabase.kt
private class RemoteSQLiteDatabase(private val binder: IRemoteSQLiteDatabase) : SupportSQLiteDatabase {
override fun beginTransaction() {
binder.beginTransaction()
}
override fun endTransaction() {
binder.endTransaction()
}
@Suppress("UNCHECKED_CAST")
override fun query(query: String, bindArgs: Array<out Any>?): Cursor {
val descriptor = binder.query(query, bindArgs as? Array<out String>)
return BulkCursorToCursorAdaptor().apply {
initialize(descriptor)
}
}
// ...and so on
override fun close() {
binder.close()
}
}
fun IBinder.asSupportSQLiteDatabase(): SupportSQLiteDatabase {
return RemoteSQLiteDatabase(IRemoteSQLiteDatabase.Stub.asInterface(this))
}
In the query method body, we receive a BulkCursorDescriptor from the callee process and wrap it into a BulkCursorToCursorAdaptor, which can be used as a normal cursor by the caller process. Now let’s do the real query in the callee process.
// RemoteSQLiteDatabaseImpl.kt
private class RemoteSQLiteDatabaseImpl(val database: SupportSQLiteDatabase) : IRemoteSQLiteDatabase.Stub() {
override fun beginTransaction() {
database.beginTransaction()
}
override fun endTransaction() {
database.endTransaction()
}
override fun query(query: String, args: Array<out String>?): BulkCursorDescriptor {
try {
val cursor = database.query(query, args)
return CursorToBulkCursorAdaptor(cursor, "name.for.cursor.window").bulkCursorDescriptor
} catch (e: Exception) {
// rethrow as IllegalStateException, which is supported by binder IPC.
error(e)
}
}
// ...and so on
override fun close() {
database.close()
}
}
fun SupportSQLiteDatabase.asBinder(): IRemoteSQLiteDatabase {
return RemoteSQLiteDatabaseImpl(this)
}
Now what? How do we open an remote database from the caller process? Try to do it yourself.
Conclusion
Now we can connect to a database that is opened in another process without ContentProvider. Those guys who love magic may need this one day.
本作品采用知识共享署名-非商业性使用-相同方式共享 4.0 国际许可协议进行许可。
本文链接：http://www.xjunz.top/post/DatabaseIPC/
