Some thoughts on community

 Hello, readers.  I come to you feeling somewhat broken, but also somewhat reflective.  I am hoping to write down some founding thoughts of a new community, having left another one that I feel has failed to live up to what I had thought were it's founding hopes.

The most important rule is this: you should always seek to encourage others to seek their dreams, and help them make good things.  It isn't necessary to explain encouragement, but is is necessary to explain discouragement.  If somebody is proposing something you think will do harm, to themselves or others, explain why you think it will cause harm.  Everyone has the right to decide for themselves how much they are willing to risk for what reward.

If you have qualifications or experience, then you have more to share.  If you wish to use your qualifications or experience to make others feel bad because they lack qualifications or experience, please leave.  "Because I said so" is not a reason for anything, and having qualifications or experience does not change that.

Tradition is an interesting thing to know about, but is not in itself a good reason to do things that way.  Often, things are traditionally done that way because it's the best way to do them.  Sometimes, it is because it was best to do them that way, but the reason it was better no longer applies.  Sometimes, doing things against tradition is a valid goal in it's own right, and sometimes, doing things against tradition is the best way of finding out if a tradition has any value.

Cartesian bots & imaging ideas

So one sort of project that I think about a fair bit, and do far less actual doing of, can be thought of as imaging with a cartesian robot.  Think of your common garden-variety 3d printer.  Now replace the plastic-squirting bits with a camera, and you've got the basic idea.

(OK, most 3d printers cheat a bit by moving the bed for the y axis, which isn't necessarily ideal, but close enough.  Now stick something you want to scan on your bed, say a PCB.  Take a whole bunch of pictures at known x/y coordinates, on a grid, do a bit of dewarping and stitching together, and bing bang bong, you've got an extremely high resolution image of your PCB.  

Further refinements: Have a ring of addressable RGB LEDs around the camera, so we can take pictures with different lighting.  Include UV LEDs and we can also catch florescence.  Add a magnometer, inductive sensor, check for capacitive coupling...

On a different tack, refine by being smart about where you take your next photo -- train an AI, or keep a level-of-detail for each image, and take an image halfway between the "sharpest" pair of neighboring images.  Should produce more useful images in shorter time then just scanning the whole grid.

Third idea, give it some sort of grippy thing.  Run k-means (or another clustering algorythm) on the images, determine your k clusters.  One of them will be "empty space".  Take the other k-1, find the best exemplar of that group (the one in the middle of the cluster).  Attempt to pick it up -- for example, increase the strength of your electromagnet until the load cell value changes, meaning you've picked something up -- move over to the largest (physical) cluster of empty space, and set your electromagnet back to zero.  Now take another photo of the region you dropped on, forcably add it to the cluster you assigned it earlier, and repeat.  (The last step *should* help with making the clusters be more orientation-independent -- that is, properties of the object that don't change when it's laying on the bed in a somewhat different position.)

If that last one worked correctly, you now have a robot that will sort your nuts from your bolts, your big bolts from your little bolts, or whatever else you care to throw at it.

On small goals, big goals, and overwhelmings

 So, I got two comments on the last bit of blogging, which is not bad considering that I only showed it to one person.  Therefore, I will now go on a bit of a rant as to why they weren't neccessirily as useful as you might have thought, Dear Reader.

So, while I'd ultimately like to automagically pull things off of the garmin gps, there's not much use in reminding me of that when I'm not in a state of "why the hell am I doing this anyway".  At present, that goal feels a very long way off, and so I need to keep it not too much in mind, for if I do I may drown in the enormity of the path ahead.

When I start on something like this, the question is where to start.  I don't want to take apart the actual widget, and doing so won't neccessarily be all that useful anyway.  It is quite small, so there's not likely to be room for such nicities as debug connectors, much less ones that are well-marked.  That means I'm going to have to try to understand it through software reverse engineering.  I like doing software re, but it's a giant topic.  Question The First: what software?  A bit of looking finds a bit of windows software that might help, the android app, and no firmware updates.  (Actually *using* the windows software, I find an update for some translation bits, but I haven't attempted to caprure it, as I doubt it would be very useful.)

The windows software appears to be written using .... can't remember the name now, and I'm doing this as a blind typing blog entry again.  Anyway, the chromium project's web-browser-pretending-to-be-a-desktop application layer.  I'm not sure if it's using that instead of electron, or if electron is really just a small layer over cwf, but I didn't see mention of electron.  I haven't looked too deeply into it as of yet, but it's also not a terribly useful app, as it doesn't actually do anything with regards to getting images/video out of the camera, only doing software updates on it.  Thus, I only gave it a fairly cursory go.

The mobile software is a bit more promising.  I haven't chedked for iOS softare -- I don't use iOS, don't know terribly much about it, and googling for android apps gives you apk download sites quite easily, whereas with iOS I'd have to go through a lot more effort.

I've had mixed luck with reversing android apps in the past.  Ghidra is my prefered method of software re these days.  It is free, wchi is a very nice price point.  It is poewrful, and it's pluggable and scriptable.  It's also open source, so if I want to know a little more detail about what it is doing and it's API, I can always just read the source.  (How useful reading the source is varies a lot.  It's a big project, and it's not always easy to follow.)  However, support for android apps is ... poor, in my experince.  It really works much better the closer to an actual cpu you are, and android executables for this sort of thing -- apk files -- are actually a zip with one or more dex files, which contain postprocessed java vm class files.  While you can load a dex file into ghidra, what it gives you isn't actually terribly useful, and it only gets worse when you have a large application that is packaged into multiple dex files.  The reasons for this are a bit hard to explain, and I'm already plenty offtopic, so I smiply won't.

This time along, I discovered jadx, or possibly rediscovered it.  Anyway, jadx lets you load up an entire apk file, and it gives you a view on it that is actually *highly* useful, with Java code for the bits in the dex, and reasonably readable Java code at that.  Whiile there are a fair few places where the dex file has been obfuscated -- removing readable names and replacing them with extremely short and meaningless ones -- there's also plenty of bits that haven't been.  Reversing the code is largely then a matter of renaming things, and it's smart enough to let you just do that by pressing the 'n' key, and it will take care itself of changing all the references to the old name.  Lovely.

Except there's a problem.  When you look past the surface layers, most of the meat just ... isn't there.  Instead, there's generated wrapper classes using libdjnni.  (Or something like that.  I can't spell so good, even when I'm not typing blind.  And yes, I can type blind surprisngly well, thouhg I am opening my eyes every now and again to make sure my hands haven't gone into off-by-one mode and started typing gibberish.)

Libdjnii is a .... dropbox/foursquare, I want to say .... library that wrapps the java native interface for use with other languages, in this case, c__ in libgecko, mostly, which brings us back to ghidra.

So, libgecko is quite large.  60,897 kB, 59 and a half meg.  For this soft of thing, that's huge, and that's the somewhat smaller armv7 version, not the aarch64 version.  That may not sound big, but for executable code compiled on a real compiler for a real cpu, and not embedding a bunch o fgraphics (as far as I can tell, it doesn't have any, and doesn't even embed user-visible text).  It does, fortunately, have a lot of names and other ELF metadata.

So, it's good that we found where the meat is.  It's good that we have symbolic names for a lot of stuff.  What isn't so good is we now have a huge mess of code to try to look through.

So big, in fact, that when I loaded it up, started going through it, using WiseRenamer to find lots of jucy details, extending it in places, I got into a state where the ghidra interface would randomly lock up for long periods of time.  I'd created too many symbols and types, I think.

I evntually ended updoing that twice over, and switched from the armv7 copy to the aarch64 copy so it'd feel slightly less like I was repeating myself.  With that one, I got to a point where it started slowing down, but not to the point where it was unusable, and started implementing some things to make it less bad.  I cleared out most of the relocation information I'd parsed, which made ghidra have a much easier time of things.  I also removeed all the generated comments, and let WiseRenamer generate most of them again.  This not only gave ghidra  a bit less work to do, it removed some comments that turned out to be more distracting to me than actually useful.

On the plus side, WiseRenamer can now do a fair bit of parsing and extracting knowledge from elf (both 32 and 64) symbols and relocations.  On the minus side, after doing much of this, I realized that I wasn't finding the information I was looking for -- what library imports came from -- not because I was failing to find it due to the layers of pointy pointers, but because it's simply not there.

See, in macho (apple) and PECOFF (windows) executables, when you use a symbol from an external library, there's a record of "the symbol named x from the library named y".  (Mostly.  Obviously I am simplifying here, and pointedly, it's possible to import by ordinal number, not name, on windows.  This isn't common, except when doing mfc.  Sadly, I end up doing mfc rather often...)  ELF binaries / linking doesn't work this way.  Instead, there's a list of "we are using these libraries", and "we want to import a symbol with this name".  However, the two aren't linked together in any meaningful way.  Instead, every time you owant to load a symbol, the system checks for a symbol by that name in ever library.  (Again, a simplifiation.)  That means I'd have to do a much more complicated, and very different, bunch of stuff to allow you to hop between import and implementation easily in ghidra.  Annoying, but not fatal, at least in this case, because libgecko is big.  In fact, it's very big.  It contains a bunch o fstuff for things like map updates, alexa integration, at least one graphics lirbary, speach recognition and generation... oh, and links to liboffnav, which I'm pretty sure is for offline navigation using your locally downloaded garmin maps.  I mean, nice of them to give us all that, but not really needed for this application, which has nothing to do with navigation.

So, yeah, a lot of stuff to look at.  So far, I've managed to identify base URLs for a whole bunch of APIs, several of which seem to be oauth stuff.  

I think that's enough for now.  You might notice that I barefly even menteiond actually running things.  That's a big topic worthy of another post, which I will probably write at some point.  Dynamic reverse engineering work (to be technical abou tit) has it's own challenges, both genreal and increasingly specific.

 Trying a little bit of external motivation, and barebones blogging.  That is, I'm going to write some stuff, possibly, as now, with my eyes closed, and post it, rather than editing and rewriting and antagonizing over every word.  Sorrry if I sometimes say something in haste and get it a little bit off.

So, I'm horrible at getting myself to work on something, and even worse about telling the outside world what I"ve been up to, so please, I would like you to leave some sort of comment on here, mentioning a project that I have ever mentioned, or just that you think I might be interested in, and I shall try to do A Thing.

Here, A Thing is defined as either writing about the story thus far on it, writing about things I am thinking of doing next, or doing some more, and writing a little bit on it.

Things I have been doing, or thingking about doing, lately include, in sno particular order, but numbered for ease of reference...

1. Bigass pan/tilt mechinisim.  I got this from a radio rally thing a while back, and I think I made it move under the controll of an arduino before lossing interest / getting distracted by oether things.  Next things would be to figure out in detail where I was at with it, and reassemble it a bit, then probably do some calculations to get it to point at either arbitrary points on the earth's surface, or arbitrary sateleities in orbit, attach a camera to it, attach an antenna...

2. Trying to extract all the data I can off of bluetooth le stuff around me.

3. Some random IoT sensors I have knocking about.

4. An old wifi / ethernet speaker / DLNA media renderer that I recently opened up and disassembled.  It has some well-marked and interesting -looking test points waiting to be soldered down to and explored.

5. Attempting to reverse engeneer the wuuk smart doorbells.  I've got a fair bit of hardware now, because I know three people who got them, three people who have had some sort of failure with them, three people who have been given a whiole new kit, and two people who have stopped using them because the faults continued.  ("People" here might be a bit of a misnomer.  Installations, including my own, as two of those three are multiperson households.)

6. A bit of an attempt to reverse engineer the garmin dashcam mini (2?, can't quite recall just now, and the rules prevent me from looking it up for this post).  Unlike the wuuk, I don't have access to hardware, at least not for the taking-apart -- we have exactly one of these, it works, and was reasonably expensive.  I'd like to keep it working, and usable.  I'd also, however, like it to nicely integrate with my local image storage, and my car pi.  Speaking of which...

7. Attempting to make a useful car pi, mostly from zero.

8. Attempting to fix the wifi in my back garden, so my car pi can have interenet when it is parked.

9. WiseRenamer, my ghidra script / dancing swiss army bell-knife.  This is my attempt to make a framework for extracting information out of executables largely "touchless", by means of reading all of the metadata I can manage.  It is currently a giant mess, because I feel very little motiviation to make it nicer.  Hopefully, writing abou tit will encourage me to do more about this.

10.  ... eh, that's enough for now, time to hit the go button.

Several Beginnings

Hello, folks!
I'm hopefully going to be writing a whole series about reverse engineering Skyrim.  I do not pretend to be an expert on reverse engineering, or video games, or much of anything, really.  Everything I say should be taken with a whole load of salt.  Product names sometimes given.  This shouldn't be construed as a recommendation.  I understand that some people might be interested in stealing things.  I'm fairly neutral on that, but I ask you to not comment on how you've gone about doing that.  The information I give won't be terribly helpful if that is your goal, either.

To follow along, you will want a copy of Skyrim: Special Edition.  As I write, it is about 20 GBP on Steam.
You will also need a copy of IDA, the interactive disassembler.  The free version will probably work fine.  The demo version should, in theory, work, but you will be unable to save, which may be ... annoying.  .  It is rather expensive.  (FIXME: Double-check what editions/versions of IDA will actually work -- we want something that will do x86-64, plus decompiler.)

So, enough with the preliminaries.  If you simply load skyrimse.exe in IDA, you won't get very far -- the .text section, where the actual code is, is quite effectively obfuscated.  After several false starts, I realized that I shouldn't be looking for a general solution, but rather one focused on Steam... and a small amount of googling found Steamless.  I've not yet managed to pierce it's inner workings, but it appears to work quite well.  I left the plugin at "automatic", and checked all the boxes.  When loading the file in ida, select "manual load" and "load resources", and say "yes" to everything -- this will preserve as much information that we might want later as possible.

(Edited 2017-02-26 for style.)