Saturday, August 22, 2020

Using Android without Google services in 2020

We all know Android is a mobile operating system used by Google as a data collection platform mainly for ad delivery. It should hardly come as a surprise if I tell you Google is mining every bit of your personal data on any average Android phone. But Android has a very important characteristic making it much more interesting than iOS for privacy minded people: its roots are buried in Open Source grounds.

Android Open Source Project

Android base is known as AOSP (Android Open Source Project). Every Android version is developed by Google, and then sources are released to the world, for any developer to browse and tinker with them. Anyone with the appropriate knowledge can download AOSP sources and (with patience and a powerful computer) build them. Add the corresponding drivers for your phone, and you will have an image you can install and use (assuming your phone has the bootloader unlocked), and that has no Google data collection services inside. There is a very active community around AOSP that will do for you the hard work of collecting the sources and building them, and also will add some more interesting features to the base. These AOSP compilations with added features are called "Android distributions" (or Android distros for short). The most well know one is "Lineage OS", but there are a lot more you can try. You can download ready to install images of these distros for a lot of phones and tablets.

The Google Apps

So far, (almost) everything is fine: you can install on your phone an Open Source OS that is not sucking your data and sending it to Google. But there is a catch: AOSP does not include the "Google Apps" or (GApps). So you will have no Play Store, no YouTube, no Maps, no Gmail, etc. If you are a privacy minded person, maybe you did not want these apps anyway, but there is one thing that is troublesome even if you do not want Google applications: you will lack Google core services, so any application trying to use them (for example any application trying to use Firebase to send a notification to the phone, or trying to use the Maps API to send your location) will lack functionality or completely fail and crash.

Google Apps are closed source. Some people are confused and think GApps are Open Source, maybe because of the OpenGapps project. This project let's you download different GApps bundles, ready to install on your AOSP based device. But you should not be fooled by the name of the project: Google Apps are 100% proprietary and we have no control on what they do. If you install them in your AOSP based device, there will be little difference regarding the data collection performed by Google with respect to the one performed by any Android phone you can buy on the market.

So if you install LineageOS on your phone (without the Google Apps), it will not collect your data and will work mostly as any other Android phone. But a lot of applications will refuse to work, because they will try to use services your phone lacks, or will directly detect you lack the Play Store and will refuse to start.

What can you do if you want a "Google-free" phone, but need some applications that refuse to work without the Google services? Here is where microG project comes to the rescue.

microG

 

microG is a free (as in freedom) re-implementation of the Google core services and libraries. It implements in a varying degree of completeness services provided by the proprietary Google Apps core, and also spoofs the Play Store so any app that requires it to be installed, can pass this check. For example, using microG, application calls to the Maps API will succeed, but instead of using the Google Maps services under the hood, they will be using Mapbox (an Open Source mapping system). Isn't this cool?

Unfortunately microG is far from perfect. First of all, it is far from completion. Some APIs are completely lacking, and most of them are only partially implemented. You can check here the implementation status. The table looks discouraging, but in my experience, most applications I tried are completely usable.

Another problem when using microG, is that even though the services implementation running on your device is free, the underlying services running on the cloud sometimes are not. If you need the device to log into Google services for using things like the Firebase messaging or SafetyNet, you will need to connect to Google machines. microG will strip identifying data other than the account name, but your device will not be 100% Google free anymore.



Finally, installing microG can be troublesome. It needs to be installed as a system application, and requires that the underlying distro supports "signature spoofing", in order for the microG Play Store application to convince other apps that it is the one coming directly from Google. Out of the box LineageOS builds do not support signature spoofing. There was a heated discussion about this topic, but LineageOS devs decided not to implement it, so something must be done here.

LineageOS for microG

If you want to get rid of Google services in your device, my advice is to install "LineageOS for microG". It is a fork of the official LineageOS distro, but with microG preinstalled (including the signature spoofing support that official LineageOS lacks). As a nice bonus, this distro also includes F-Droid preinstalled, a substitute of the Play Store that has only Free and Open Source Software (FOSS). First you have to check your device is supported. Then for the installation, instructions will vary depending on your device. You will have to check places like XDA-Developers to get them, or even better, find a friend that can do the work for you.

Once you have LineageOS for microG, or any other AOSP based distro with microG on your device, you will have regained total control of it. You can connect to Google servers only if you really need them, and in my experience you will not need to connect at all if you are ready to make some sacrifices. Also no matter what you do, some applications will refuse to work, so you will have to search a substitute (F-Droid is a great tool for this) or just live without them, that is if you are resolved to not use Google services anymore. I use LineageOS for microG on a daily basis, and for me it is pretty usable, I will try documenting some of the things I use and work, and some of the things I lost in this adventure.

What works

Unless otherwise stated, all the applications I will comment here, either come preinstalled or are downloaded directly from F-Droid. If you need downloading an application from the Play Store, there are several methods that work (setting a local repo, using browser plugins, some special apps, etc).

Where's my bloat?


  • "Standard" smartphone tools (calls, SMS clock and alarms, taking photographs, videos, gallery, etc.) work perfect out of the box. You will have some added bonus you will hardly find on any phone you can buy on a shop, such as the ability to record calls.
  • Contacts and Calendar synchronization: I use them with Nextcloud (with DAVx5 app) and they work perfect.
  • e-mail: The included email application works great. In addition to implementing the standard e-mail protocols (SMTP, POP3), it can interact with Exchange (Outlook) and also with Gmail accounts. The only drawback is that if you use second factor authentication, you will need to set up a password for this application. But this is a one-time hassle.
  • Messaging: Telegram works just perfect and does not require connecting to Google services, this is the app I would recommend for messaging. There is an old Telegram version on F-Droid, but if you want a recent one, you will need to get it from the Play Store or elsewhere. Some years ago I used WhatsApp and it also worked, but Firebase/Google Cloud Messaging was needed to get real time notifications. Also WhatsApp is closed source and has ties to Facebook, so if you are still using it, my advice is to uninstall it ASAP. Of course WhatsApp can only be download from the Play Store.
  • Mapping: I use OsmAnd+. It is not as great as Google Maps, but it has many features and is a pretty good substitute.
  • Browsers: Firefox works just perfect. I have not tried Chrome, but I think it also works great. Chrome is not available in F-Droid. Neither is Firefox, but in this case you can download in F-Droid a Firefox updater app that will fetch the most recent version for you.
  • Cloud synchronization (files and others): All the Nextcloud apps I have tried work perfect (Nextcloud, Bookmarks, Notes and Deck).
  • TOTP password generation: andOTP works great and has a lot of options to securely store your OTP configurations.
  • YouTube: I have not tried the official app, maybe it works. But I use NewPipe to watch YouTube videos. It works great most of the time and also has some added features, like downloading videos or reproducing them on a popup window. Make sure you have it up to date, because it sometimes breaks (when Google makes changes to YouTube internals) and developers are quick to fix it.
  • Twitter: I have not tried the official client, I use Twidere app. It is lightweight and full featured, I really like it. The only thing I lack in this app is that it does not yet support Twitter polls. Other than this, it is just superb.
  • Reddit: I use Slide app, and it works great. 
  • Games: All the games I tested downloading them from the Humble Store worked perfect without exception. I have tried at least Monument Valley, Plants vs Zombies, Canabalt HD, Super Hexagon, World of Goo...
  • Emulators: I have only tried RetroArch and it works perfect. Unfortunately it is not available in F-Droid (even though it is Open Source).
  • Multimedia playback: just install VLC. It can play any file you are able to throw it. If you have a media center based on Kodi, both official Kodi app and also Kore app work perfect (I personally prefer the later). I also use TVHGuide to view TDT TV emissions directly streamed from my media center (that has a TDT decoder and Tvheadend software).
  • Spotify: the official Spotify app works perfect. You will not find it in F-Droid, as it is closed source.
  • QR Scan: SecScanQR works perfect and also supports standard linear barcodes.
  • Passbook handling: PassAndroid works perfect.
  • Podcasting: just install AntennaBox.
  • PDF readers: PDF Viewer Plus works great.
  • Dictionary: WordReference (not in F-Droid) works great. I have yet to try some of the Open Source dictionary apps in F-Droid.
  • Rooting: If you need root permissions, Magisk works like a charm. It is not in F-Droid (you have to flash it from the Recovery). Say goodbye to passing SafetyNet checks if you are going to install any root application, including Magisk.
  • Backups: I use TitaniumBackup and it works perfect. Even the license works without having the Google Apps (you have to ask the developer for a license key file). This app is proprietary and as such it is not in F-Droid.
  • Networking: OpenVPN works perfect.
  • Swype type keyboard: AOSP keyboard already supports swype typing style. But unfortunately, for this to work it requires a closed source library (libjni_latinimegoogle.so). You can grab this library from the OpenGapps package and manually flash it for this function to work. Otherwise you will lack this function.

What doesn't work

Here are the things I found not working, and some more comments about things I have not tested but I suspect will not work.

  • BBVA España: this is a spanish banking app. It warns the device OS has been modified and then crashes.

Others I have not tested but I think that have many chances of not working:

  • Other banking apps. I suppose there are many chances for them not to work, or at least to force you to connect to Google (and not have root) to pass SafetyNet checks.
  • Applications requiring DRM media playback (like Netflix). I have not tested but I would be surprised if they work.
  • Android Auto. API has not been implemented, so do not expect it to work.
  • Some games using the Games API (but I had no one to test).

TL;DR

For enhanced privacy, it is possible to use an Android phone lacking Google Services. Install LineageOS for microG (or any other Android distro, and then microG) and most apps should just work. Unfortunately a few of them will fail with this setup, so if you can find alternatives, great. Otherwise, you have to balance what you prefer: the enhanced privacy or the app you cannot use.

Wednesday, May 6, 2020

MegaWiFi, and why you should code for it

It's been a long time since I wrote here about MegaWiFi. To date, there are still no games released using it, so you might think this project is long dead, do you?

Well, if you do, you are wrong! Progress has been slow (my spare time is limited), but lot of things have happened:

  • MegaWiFi API has greatly matured (currently it is version 1.3), and I spent a lot of time working on making it easy to use. I have created an implementation of pseudo threads and timers, allowing to use the APIs in both synchronous and asynchronous flavors. It has support for TCP, UDP, SNTP, HTTP/HTTPS, etc.
  • I also spent a lot of time working on API documentation. All API functions are pretty well documented, and I have provided properly explained examples for most common use cases.
  • There was an initial effort long time ago to add MegaWiFi support to the great Blastem! emulator. Unfortunately this was halted prior completion. But recently I have resumed work on this, and now MegaWiFi support in Blastem! is very usable.
  • There is a game finished and ready for release (WorldCup 1985 by Pocket Lucho) with MegaWiFi support! It should be released soon by 1985alternativo. Go to my previous post to reach the repos
 
Megadrive/PC cross play, what a time to be alive!
  • There is at least another game with MegaWiFi support in early development, again by Pocket Lucho.
  • I added support to use MegaWiFi with the popular SGDK development kit, with detailed instructions and a precompiled toolchain for Windows, ready to use.
  • I wrote a Gaming Channel implementation, allowing devs to publish games online, and users downloading them directly to the cart. This shows the platform is rock-solid stable, allowing to flawlessly download and play games.
 
Steam is something from the past!

  • And there are many many more features I will not bring here now, to avoid making this post too long.
So, to sum it all, if you like coding for these vintage devices, you should consider starting coding a WiFi enabled Megadrive game NOW! The reasons?:
  1. Hardware and Firmware are mature.
  2. API is mature, well documented and easy to use.
  3. You can code your way or use SGDK for an easy kickstart.
  4. There are great development tools readily available, and emulator support, including integrated debugger with breakpoints, variables watch, etc. Yes, Blastem! is great!
  5. You should be able to publish the games in physical cartridge format, make them available as digital downloads directly to the Megadrive, or play them via emulator.
  6. Everything is Open Source Software and Hardware: the cart and programmer schematics, the firmware for the WiFi module and programmer, the API for Megadrive, the PC tool for programming, the WiFi bootloader...). Go to my previous post to reach the repos.
  7. Megadrive is cool, and WiFi on the Megadrive is even cooler!