Sync using OneDrive

MMEX4Android: Manage your finances on-the-go, encrypted for security, sync via your own cloud

Moderator: Renato

wizardict
New MMEX User
Posts: 14
Joined: Fri Aug 26, 2016 3:27 pm
Are you a spam bot?: No

Re: Sync using OneDrive

Post by wizardict »

GabrieleV wrote:You are right that a centralized DB will solve problems, but most of MMEX users don't want to have their data on a webserver not under their control, so the only way it's keep in sync the full DB file..
That's a good point. I wasn't thinking of a hosted server, more a database that is local to the user so they still have full control, but still that would not suit everyone.

There could be a solution that would suit everyone though, something like SymmetricDB.  That would allow local sqllite databases which you have now, but with the option to sync to a central server for those who want it. It does proper SQL sync so would allow multiple users to use the application at the same time, and would also still allow the mobile app to be used offline.


If anyone is interested in this kind of solution I would be prepared to help develop the desktop side, but it would need your Android developer to implement the android version (which is documented on their website) as I have no experience in android development unfortunately.
User avatar
GabrieleV
MMEX Developer
Posts: 158
Joined: Wed Apr 09, 2014 7:45 pm
Are you a spam bot?: No
Location: Italy

Re: Sync using OneDrive

Post by GabrieleV »

Is this one? https://www.symmetricds.org

Considering case where there isn't a central server for security reasons, how will sync handled?
There will be for example 3 DB files on Dropbox called "master", "android", "desktop" and symmetricds will keep them in sync?
wizardict
New MMEX User
Posts: 14
Joined: Fri Aug 26, 2016 3:27 pm
Are you a spam bot?: No

Re: Sync using OneDrive

Post by wizardict »

GabrieleV wrote:Is this one? https://www.symmetricds.org

Considering case where there isn't a central server for security reasons, how will sync handled?
There will be for example 3 DB files on Dropbox called "master", "android", "desktop" and symmetricds will keep them in sync?
Yes that is the correct product, as you can see it is open source so fits with your product nicely.


The intention of this would be to allow people the option to include a central server, not exclude it. People who don't want a central server would continue to work as they are, people who do want improved functionality and proper sync functions would use this. So...


They would set up a central server of their own (it could be on their desktop still, or a web server whatever they prefer. It could by MSSQL, MYSQL or even SQLITE still, whatever they prefer and is supported by symmetricds).

All of the installs (desktop, mobile client etc) would store the data locally in their own SQLite database, not on dropbox or OneDrive etc these are not suitable methods to sync databases and aren't intended to be used in this way.

SymmetricDS would be used to sync each client with the "central server" owned by the user.




Synchronisation would be far improved, without the "single client only" model that is al that's possible with the current system. This would also open the possibility for different client development, such as a fully functional web based client. As soon as you have a proper "client > server" model you open up a lot of new possibilities and functionality, but doing it this way would still allow people to have the simple standalone install they currently have if they prefer.
User avatar
MisterY
Super MMEX User
Posts: 137
Joined: Sat Jan 24, 2015 8:11 pm
Are you a spam bot?: No

Re: Sync using OneDrive

Post by MisterY »

wizardict wrote:This is starting to feel like my personal blog! Does anyone involved in development monitor or reply to posts on here?

There's a new release of the android app, but unfortunately the sync issues remain. Entities do not sync to the app, and any changes made on the mobile app uploads the database and delete all of the changes that haven't been synchronised.

There is also the issue of the mobile app causing a new database to be created (i.e. it doesn't update the database it results in a separate database being created). I see this issue is reported elsewhere on the forum with dropbox too, so it isn't OneDrive specific.



Unless there is a response from development that these issue are being looked in to I think I will have to abandon MMEX and look elsewhere. It's shame, as it has promise, but without a working mobile version I don't think this is a viable project, I'm happy to help trying to resolve these issues, but with no response I don't really know who I'm trying to help.
Hi,

As the very first item, I would kindly point you to read the documentation that was written for this purpose. It might not be "in-your-face" type of documentation but there is a link to the User Manual from the app's home page. There the process of setting up the synchronization is explained into some detail. There are also a few more pages referring specifically to the (previous) Dropbox-only synchronization but their value is in explaining the concepts. I mention this because some people open the database file from a Dropbox client and expect the synchronization to work seamlessly.
Secondly, please note that this is an Open Source project and it is nobody's duty to supply support. You are certainly welcome to contribute functionality, documentation, code, translations, and provide support to other users in the forums, just like we all are.

If you experience specific issues, you can report them through several channels but do please keep in mind that we also have a life and full-time jobs that take priority over any hobby project. Do get in touch via email, or post an issue on GitHub.
User avatar
MisterY
Super MMEX User
Posts: 137
Joined: Sat Jan 24, 2015 8:11 pm
Are you a spam bot?: No

Re: Sync using OneDrive

Post by MisterY »

wizardict wrote:OK.....I have located the root of this issue.

The problem occurs when the mobile app can't update the database (i.e. it is locked by the desktop client) and a 2nd database is created. As long as the 2 databases exist OneDrive does not update them correctly, so the 2nd database needs to be deleted then everything syncs as expected.


So the issue is with conflicting updates. Do Dropbox and/or Google drive handle conflicts in the same way? I've seen the same issue reported on dropbox, so presumably that's the same, what about the others?

All of this would be resolved with a proper database and network capabilities :P
Hi, sorry I replied before reading the whole thread.
Yes! This is one of the key issues for the desktop version. The database file is locked as long as the application is running. This will **always** cause a sync conflict if the file is modified on mobile at the same time.
It has been noted earlier and I'm not sure if there is something that can be done in the application, except perhaps constantly closing the connection when it is not needed.
**Or**, a user could simply close the application when he's done with it.
wizardict
New MMEX User
Posts: 14
Joined: Fri Aug 26, 2016 3:27 pm
Are you a spam bot?: No

Re: Sync using OneDrive

Post by wizardict »

MisterY wrote:
wizardict wrote:OK.....I have located the root of this issue.

The problem occurs when the mobile app can't update the database (i.e. it is locked by the desktop client) and a 2nd database is created. As long as the 2 databases exist OneDrive does not update them correctly, so the 2nd database needs to be deleted then everything syncs as expected.


So the issue is with conflicting updates. Do Dropbox and/or Google drive handle conflicts in the same way? I've seen the same issue reported on dropbox, so presumably that's the same, what about the others?

All of this would be resolved with a proper database and network capabilities :P
Hi, sorry I replied before reading the whole thread.
Yes! This is one of the key issues for the desktop version. The database file is locked as long as the application is running. This will **always** cause a sync conflict if the file is modified on mobile at the same time.
It has been noted earlier and I'm not sure if there is something that can be done in the application, except perhaps constantly closing the connection when it is not needed.
**Or**, a user could simply close the application when he's done with it.
Hi

Thank you for your reply. If you read a little further you will se I have suggested a solution, which I will happily contribute towards, but isn't something I could achieve on my own. Do you have any thoughts on the Client>Server model with synchronisation suggested? The reality is dropbox etc were never intended to be used in this way, so to develop an application that requires sync across multiple clients really means moving away from this model.
Last edited by wizardict on Thu Sep 01, 2016 10:38 am, edited 1 time in total.
User avatar
MisterY
Super MMEX User
Posts: 137
Joined: Sat Jan 24, 2015 8:11 pm
Are you a spam bot?: No

Re: Sync using OneDrive

Post by MisterY »

wizardict wrote:
GabrieleV wrote:Is this one? https://www.symmetricds.org

Considering case where there isn't a central server for security reasons, how will sync handled?
There will be for example 3 DB files on Dropbox called "master", "android", "desktop" and symmetricds will keep them in sync?
Yes that is the correct product, as you can see it is open source so fits with your product nicely.


The intention of this would be to allow people the option to include a central server, not exclude it. People who don't want a central server would continue to work as they are, people who do want improved functionality and proper sync functions would use this. So...


They would set up a central server of their own (it could be on their desktop still, or a web server whatever they prefer. It could by MSSQL, MYSQL or even SQLITE still, whatever they prefer and is supported by symmetricds).

All of the installs (desktop, mobile client etc) would store the data locally in their own SQLite database, not on dropbox or OneDrive etc these are not suitable methods to sync databases and aren't intended to be used in this way.

SymmetricDS would be used to sync each client with the "central server" owned by the user.




Synchronisation would be far improved, without the "single client only" model that is al that's possible with the current system. This would also open the possibility for different client development, such as a fully functional web based client. As soon as you have a proper "client > server" model you open up a lot of new possibilities and functionality, but doing it this way would still allow people to have the simple standalone install they currently have if they prefer.
I like the idea in general but wouldn't this destroy the option of having a standalone application, whether desktop or mobile? There's still plenty of users who do not synchronize the database and use a single platform exclusively.
User avatar
MisterY
Super MMEX User
Posts: 137
Joined: Sat Jan 24, 2015 8:11 pm
Are you a spam bot?: No

Re: Sync using OneDrive

Post by MisterY »

wizardict wrote: Hi

Thank you for your reply. If you read a little further you will se I have suggested a solution, which I will happily contribute towards, but isn't something I could achieve on my own. Do you have any thoughts on the Client>Server model with synchronisation suggested?
There would probably be an option to initialize this only as a part of synchronization settings?
wizardict
New MMEX User
Posts: 14
Joined: Fri Aug 26, 2016 3:27 pm
Are you a spam bot?: No

Re: Sync using OneDrive

Post by wizardict »

MisterY wrote:
wizardict wrote: Hi

Thank you for your reply. If you read a little further you will se I have suggested a solution, which I will happily contribute towards, but isn't something I could achieve on my own. Do you have any thoughts on the Client>Server model with synchronisation suggested?
There would probably be an option to initialize this only as a part of synchronization settings?
Yes exactly. The application would work exactly as it does currently, with data saved to a local SQLite database, so people that are happy using it this way can continue to do so.
For people that require greater network capabilities there would be an option in the settings to configure the "master server", which would then sync the data to that central database. If you don't want to use it, simply don't enable it.
User avatar
MisterY
Super MMEX User
Posts: 137
Joined: Sat Jan 24, 2015 8:11 pm
Are you a spam bot?: No

Re: Sync using OneDrive

Post by MisterY »

wizardict wrote:
MisterY wrote:
wizardict wrote: Hi

Thank you for your reply. If you read a little further you will se I have suggested a solution, which I will happily contribute towards, but isn't something I could achieve on my own. Do you have any thoughts on the Client>Server model with synchronisation suggested?
There would probably be an option to initialize this only as a part of synchronization settings?
Yes exactly. The application would work exactly as it does currently, with data saved to a local SQLite database, so people that are happy using it this way can continue to do so.
For people that require greater network capabilities there would be an option in the settings to configure the "master server", which would then sync the data to that central database. If you don't want to use it, simply don't enable it.
Certainly seems interesting. I'll do some more reading through their documentation. And would certainly love to use it. Especially if it
could do a record-level sync. This would  greatly reduce the size of the network traffic, for example.
However, I'm concerned that most users would have great issues installing their
own server. Even supporting the sync via well-known file-syncing
providers is causing too much trouble already.

Going back to one thing you mentioned, that you don't know much about Android development.
Well, neither did I before joining this project. It was a way to learn
more and get my hands dirty, so to speak. In any case, I wholeheartedly
recommend the following video to everyone. Please watch at least the
first five minutes:

https://www.youtube.com/watch?v=TK1mBqKvIyU
Post Reply