Showing posts with label tips. Show all posts
Showing posts with label tips. Show all posts


CrashPlan limitations

After a few weeks using CrashPlan PRO for small business (not the free tier!) I tried contacting support to ask how to do some underexplained things, and/or to open some bug reports/feature requests. Like:
  • How to know which were the last backed-up files? or, when was a file last backed up? (maybe some useless-but-frequently-updated file is slowing down backups of bigger and more important files?)
  • The app defines backup sets, but they don't seem to correspond to "restore sets". How to restore a particular set?
Support's answers were pretty underwhelming, bordering on canned-responses; but I found that Code42, the makers of CrashPlan, have an API to access the backup data. So I browsed a bit to see whether it'd be easy to do myself what I wanted.


Review of air purifier Prem-i-Air Invierno and air analyzer AirVisual Node

Living in Warsaw, I got tired of worrying about the air quality – which some days during winter got reported as worse than that of infamously polluted Beijing. So some months ago I bought an air purifier.

I had a number of doubts about how to use it so it was effective: does it have to work all day? The purifier's power level depends on the room's size, how do I know that it's working as it should? Air quality sensors included in these purifiers are said to be pretty unreliable, are they any useful at all? How does air quality indoors correlate to air quality reported by an official station about 1 km away from home – which daily recommended wearing a mask during winter!? What happens if I just leave the windows closed, or if I open them for a while? ...

So I bit the bullet and bought an air quality sensor, which cost almost as much as the purifier itself.
And these are my results and tips for anyone thinking about doing the same.


User-site-installed python packages, and PATH modification

PEP 370 allows you to have user-local installs of packages. You're supposed to run pip --user install whatever.

If you want to do that by default, so any future "pip install" does install to the user-site, you can add a pip configuration file telling pip to do so. In MacOS, the file must be located at ~/Library/Application Support/pip/pip.conf, and contains
user = true
(BEWARE! If you use virtual environments, note that thanks to another instance of Python brain damage/unfinished business, as of Python 3.5.3 and pip 9 the --user argument causes problems with installs in virtual environments of both venv and virtualenv varieties... and looks like the problem comes rather from the argument, not from the functionality! A Quick&Dirty fix is to do something like export PIP_CONFIG_FILE=/dev/null inside the virtual environment. This could be automated with hooks in virtualenvwrapper, but it starts to feel like a pile-up of hacks...)

OK, so now you have user-local installs by default. But their commands are not in your PATH, so you need to set that too.

You could go the naïve way and just add something like
to your .profile. But, note: here we're hardwiring python 3.5 (or any other version). Is that what you want? Is the rest of your system so hardwired? Unless you specifically did so, the answer probably is: no, it isn't. So you might end up mixing Python installs/environments.


Installing Ubuntu 16.04's friends into an iBook G4 with a USB pendrive

I had an old iBook G4 that could be useful for an elderly person, to be able to do her banking and basic internet browsing without having to use an XP-vintage, molasses-slow, honeypot-aspiring PC.

There's a reasonably complete and up-to-date guide in I had to change a couple of details; and there are a couple of practical considerations to be had.

(Note to self: "lightness" in Linux rather correlates to "barebones" [3], which correlates with "hardcore" (text configs, mailing list scouring, lots of googling). Next time I'll be trying a heavy-weight distro for this kind of thing; better slow-but-useable than less-slow-but-WTF.)


Cocinando con citrato sódico casero

Parece ser que un ingrediente típico para la "cocina molecular" de Ferrán Adriá y compañía es el citrato sódico, que se usa por ejemplo para esferificaciones, caviar de frutas y cosas así.

Pero también se usa para algo mucho más simple y usable en casa en el día a día: ¡queso fundido!, que queda genial para salsas, dips, incluso fondues.

¿Pero cuál es el problema? Fundir queso es fácil, ¿no? En realidad, no. Prueba a fundir un taco de 100 gramos de queso normal y verás que el resultado es casi para tirar a la basura: un charco de aceite por un lado, una especie de trapo gomoso por otro. Esto lo podemos salvar con citrato sódico.


Doxygen error parsing in Eclipse CDT

Eclipse (or is it the CDT?) has, by default, some Doxygen integration: if the preferences are set appropriately, Doxygen-style comments are highlighted differently, and Doxygen commands in those comments are further highlighted.

But that's about it. A particular missing feature is that errors in the comments are not highlighted in any way; and that's what made me look for a better alternative.


repo sync --force-sync overwrites your existing repository!

I couldn't find any good explanation of what does Android's repo's --force-sync do, nor why it can be needed. So I'm reporting my painful findings in repo 1.22.


Best practices for a (GNU) makefile in the XXI century

This is not really a tutorial on makefiles; there are lots of those around the web.

But most of those are very outdated, and/or follow dubious practices. You'll end up with a makefile which was OK for a make and a compiler from the 90's, maybe even around year 2000. Even the GNU make manual recommends unnecessarily complicated things, given the capabilities gcc has grown in the last decade.

So, if you know how makefiles generally work (it's enough to know what a rule, a recipe and a variable is), and want to improve yours or know of better options, the following may help.


git mental model

Finally I managed to get a mental model of git that allows me to do everything without needing to google around. And this is a quick memory dump.

There are other git cheatsheets, but this is mine.


Modelo mental de git

Por fin conseguí hacerme un modelo mental de git que me permite hacer de todo sin necesidad de google ni esquemas ni Stack Overflow ;P. Y ésta es mi chuleta / "volcado de memoria". Parece que no hay muchas cosas así en español; a ver si le sirve a alguien.

La idea era mantenerlo muy corto y al grano — como una chuleta, vaya. Asumo que sabes de qué va en general git y qué son cosas como un puntero y una lista simplemente enlazada. Si hay interés lo expandiré en algo más "tutorial".


6 tips to survive Codility tests

Well, it happened. When applying for a job, I got sent to a Codility test. And even though I guess I already had some good practice with them, I managed to do badly in stupid ways – ways that 2 years ago I had already thought about and even taken notes on. Just, in the heat of the moment I forgot about those rules of thumb.

And in fact I think these hints should be given by Codility themselves – because if not you are practically ambushed, even more so if you didn't take the time to thoroughly explore how they work. So here are my hints-to-self.

The summary is: don't think of this as a coding interview; this is rather about getting something working, ASAP.


Damaged hard disks and data salvaging: stay well away from SpinRite

This is something I posted to MacIntouch on 2011; reposting here hoping it's easier to find and helps someone.

(Interestingly I made a spanish version of this post which has attracted a couple of True Believers. I'm looking forward to see what happens in english)


Eclipse CDT configuration for big, makefile-based projects

It's been kinda hard to get Eclipse to work well with big CDT projects (say, the Linux kernel, or even a BuildRoot project including the Linux kernel). The documentation for Eclipse is between non-existent and unhelpful, and I only could find a post that talked specifically about preparing an Eclipse project for the Linux kernel. But that post explains nothing, and does some arbitrary things. So this is the generalization of that.

Telnet autologin with a short Expect script

Couldn't find anything concise, so here's my solution: a sweet short Expect (the CLI variety installed by default in Ubuntu) script to open a telnet session, feed in the user and password, AND continue to an interactive session.


Stopping OS X's from storing locally IMAP messages

TL;DR: wants you to download your messages even if you change manually the account .plist. Forget about it.

I wanted to stop in OS X 10.10 Yosemite from caching locally my IMAP messages. Once upon a time there was right there in an option to do just that, but for the last couple of OS versions the only option shown there is whether to download the attachments or not.


Using OS X’s syslogd to receive log messages from the network

TL;DR: avoid this buggy mess and go with macports & syslog-ng. You'll finish faster and saner.

[Updated 2 times]

This sounds like should be easy, but OS X is a moving target because of all the infrastructure changes they have been making for the last few OS releases. Yes, there is a syslogd, but it is some half-hollowed out thing and “others” do most of the work. Syslogd does NOT open an UDP socket, launchd does and feeds it to syslogd. Syslogd does NOT (really) receive the UDP packets, a plugin does it. Syslog does NOT parse the UDP message, ASL (Apple System Log?) does. Syslogd does NOT filter the messages and store them into “logs”, ASL does.

So why there is a syslogd at all, apart from giving a slight sense of false security? (As in, “c’mon, there’s syslog, can’t be too difficult”). No idea. If I had seen how complicated this was going to get I would have bailed out and used syslog-ng from macports.

Anyway. So the first step is to enable UDP reception. The manpage for syslogd explains what has to be changed and where, but doesn’t mention the intermediate step of converting the syslogd plist into xml. Which is easy with plutil or by using some GUI editor, but the fastest way is to just change things from the command line:
cd /System/Library/LaunchDaemons
sudo /usr/libexec/PlistBuddy -c "add :Sockets:NetworkListener dict"
sudo /usr/libexec/PlistBuddy -c "add :Sockets:NetworkListener:SockServiceName string syslog"
sudo /usr/libexec/PlistBuddy -c "add :Sockets:NetworkListener:SockType string dgram"
sudo launchctl unload
sudo launchctl load

With that, log messages originating from the network will be dumped into the normal ASL message storage. Some of that goes into BSD-style, flat text files. But another part of it does not, and stays in some internal ASL database. So the best way to see everything kind-of-at-once is to use the OS X Console utility.

However, the UDP syslog protocol is supposed to include in the UDP packets some information that is not appearing in my Console. Most importantly, there should be a Host field, which would be perfect to classify the log messages coming from the network; alas, in my usage I am not seeing that. After checking the debug info from syslog and ASL and their dogs, looks like either the Linux side (DD-WRT) is not sending the packets correctly or the Mac side is not parsing them right (no, I didn’t feel like trying Wireshark), and some fields are either swapped or missing-and-not-accounted-for, causing some fields to contain values that should be in other fields. Which results in the log messages coming from the network getting lost among the local log messages… and there is no direct way to distinguish them.

So, next idea: dump the log messages coming from the network into a separate file.
To do this, it’s necessary to add a rule to ASL so it filters out the messages that came from the network and writes them into our file.
The problem is that once the messages are inside ASL, already there is no direct way to know where they came from! Luckily, they can still be indirectly recognized because of some anomalies: for example their GID, UID and PID are nonsensical and always the same: GID = UID = -2, PID = -1.
(note that the GID, UID, PID and other keys are not usually in the BSD-flat-format typical in logs; but ASL does store metadata for every message, and a lot of that metadata is simply ignored when outputing to the BSD format. So the idea of writing the UDP-received messages into a file is not the best solution, and I would have preferred being able to continue using some ASL storage and to comb through it; alas, I couldn’t find a way)

After some tests, this is what worked for me: put the following into the file /etc/asl/syslog_UDP :
> my_syslog.log mode=0740 format=std rotate=seq compress file_max=5M all_max=50M gid=20
? [N< PID 0] file my_syslog.log

And signaled syslogd to reconfigure:
sudo killall -SIGHUP syslogd

Aaaand it’s working! In /var/log a folder named syslog_UDP will appear, in which logfiles named my_syslog.log will contain log messages with a PID lesser than 0, which should be impossible - but that is what the UDP-originating messages report. Note too that the logfiles will be automatically rotated, compressed and purged after they start taking more than 50 MB. And they are readable by everyone - no sudo needed. (You are not running as an admin, are you?)

This will break when either DD-WRT starts sending correct UDP messages, or when OS X starts parsing them right, or when the syslog/ASL mechanism changes…

One thing that I tried and couldn’t get to work is using “directory my_syslog.log” instead of "file...", but the result seems to be buggy, in that the generated files inside the directory do not honor the mode or UID/GID that I try to set.
And the debug support in ASL (activated by adding a line “= debug 1” in /etc/asl.conf) is less than useful. Finally what helped the most was to use the message inspector in and the format=raw output flag to see the key values as they are at filtering time - the -1 and -2 values are in fact shown as the decimal value of 0xFFFF and 0xFFFF+1 in the message inspector, which did not work when filtering

In hindsight I’ll bet that using syslog-ng, or even some hackish reverse netcat, would have been much faster. Buuut… well, at least I have learnt a bit about the logging madness in OS X, which in fact had been kind of a to-do for some years…


UPDATE 3/3/2015: Beware, this seems to stop working after a while. My first troubleshooting shows OS X's syslogd to be buggy when receiving from UDP (in Yosemite at the very least): syslogd stopped working (seemingly at all, not only the UDP reception) after some hours, and lsof shows that syslogd had 255 file descriptors open, most of them to the UDP socket. Sending a SIGHUP didn't do anything. "sudo killall syslogd" restarted it and now seems to be working, but who knows for how long. So I see myself going the syslog-ng/macports route in the near future.

UPDATE 4/3/2015: the UDP file descriptor growth keeps happening. Seems to be related to my WiFi connection going on and off. Nastily unreliable, and a funny way to remotely disable syslog on an OS X system, too. I'll disable the UDP thing and send a bug report. My recommendation: just go with macports & syslog-ng.


Selecting a mobile internet operator in Poland - 2014 edition

2 years ago I wrote about selecting a mobile internet operator in Poland, and the post seemed to be popular; so here's an update.

Once again I am looking for better / cheaper alternatives to T-Mobile's mobile internet, given that my usage is going down.


Salted caramel sauce with muscovado sugar

I wanted to try making (salted) caramel sauce with muscovado sugar, but I couldn’t find any hint if it was even possible; the most I could find was someone supposing that it would be impossible or inedible because the molasses would get burnt, and some recipes that included some part of muscovado sugar into a much bigger quantity of normal caramel.
But I wanted only muscovado sugar!, so I decided to try and document the experience.


Filing the PIT from Mac OS X

If you are security-conscious, you probably have limited all kind of things that Adobe Reader can do (Javascript, opening links, …). Probably also in you browser configuration (plugin activation or even desinstallation), you OS configuration (limiting which programs can run and when, uninstalling things, etc).

You might even have a Mac. ;P

But then some program comes around which just assumes things to be standard and doesn’t try too hard to adapt to anything. Like the e-Deklaracje program used for filling in the tax documents in Poland (PIT-37 in my case). Made with Adobe AIR, and needed (?) to minimize the red tape, so the fuckers can count on the user doing whatever is necessary to appease the monster.

The first couple of years I used a Virtual Machine with Windows XP for those things. But already for the last couple of years I have managed to do everything in Mac OS X; I only have to remember all the little tweaks that have to be done or undone so the e-Deklaracje program runs successfully. So here's the (partial?) list.

So, the thing is every option has to be set into pretty much the most permissive possibility. Note that there are so many options along the way that checking how they interact would be too much work; so I am not totally sure that everything I list here is needed. I only know that when all of this is set like this, e-Deklaracje does work.

Make sure that the Adobe Reader plugin is installed in /Library/Internet Plugins; it surely got there when you instaled Adobe Reader, unless you specifically went in to delete or disable it, as I had done.
(One note here: looks like once you install Adobe Reader without the Internet plugin, the normal installer will refuse to run, saying that you already have installed the application. So if you want to install the plugin, you have to either get ride of the installed Reader, or download the full Adobe Reader offline installer)

In Adobe Reader’s preferences, enable JavaScript, and in the Internet section, enable the plugin - if that option appears. This is “funny”, because sometimes you are presented with a checkbox, but sometimes with just some descriptive text. I am not sure what causes the change. Right now for me it shows the text, saying that “the browser will use Acrobat Reader”.
In the Security (Enhanced) section, I have it enabled.
(Want to have a chuckle? There is a link right there saying “What is Enhanced Security?”, which takes you to the browser, where you are warned that you are opening a page at with a self-signed certificate for “localhost”!)

In the Trust Manager section, in Internet Access check your settings. e-Deklaracje is shown there as allowed to do what it pleases, I don’t know if I was asked about it some past year or if it just put itself there somehow. Also make sure that if a PDF tries to send information to the net, it will ask, not block unconditionally.
In the Forms section I have “Automatically calculate field values”, and to show everything.

And finally make sure that Safari has all its plugin and security settings allowing Acrobat Reader to do its thing: I had to enable the plugin in the Plugin Manager in the Security preferences (it had been disabled by Safari because it does not allow for the highest security settings). Note, even if you don’t use Safari this can be necessary, since Safari does supply in fact some flavor of OS-wide internet settings.

…and I think that is all.

Just remember to lock down again everything when you finish! Personally, I disable Javascript in Adobe Reader's preferences, and manually move the AdobePDF* files from /Library/Internet plug-ins into a folder I made named "Internet plug-ins DISABLED". That way the PDF viewer native to each browser works again.

One more thing: The e-Deklaracje application doesn't seem to be really necessary, though it helps you manage other forms and all the process and history. All the filling-in of the forms themselves can be done by plain Acrobat Reader, so you only need to get the PDF document for your PIT. I am not sure about the sending step, but I seem to remember from other years that it could be also done either from Reader itself, or from the e-Deklaracje website.

EDITED on 2015: an earlier version of this post mentioned some extra settings, but I deleted them when I could ascertain that they are not necessary for e-Deklaracje to work.


Estimation questions in job interviews

I had heard about those questions in job interviews where you are asked to estimate on the spot something, though having no real data about it.

There was some example about estimating the quantity of gas stations on the whole USA; I seem to remember that was for a Google job application. The guy had the guts to throw up numbers in a hunch with the handwaviest idea of how they could be good numbers… and, luckily, at the end the result was even not a bad approximation.

I realize now that at the moment, when I read that, the “luckily” part is what stuck on me. While it’s true that the guy had the wit to back the numbers and/or the chutzpah to pick them out of thin air, and even though he also had some brillian moments to just turn around and cross-check some imagined number by using another set of assumptions… the thing is that finally he was near the true number.

It was fascinating, and rather intimidating. How in hell would I be able to do something like that?

So, when in a recent job interview I was asked point-blank for such an estimation, I was totally caught off-guard. I was rather well prepared technically, if only because I had been kind-of-practicing lately - but I wasn’t really expecting any more technical questions that day (it was late after work!), and certainly no estimations. But here I was, with the interviewer just charmingly dropping the question and handing me a piece of paper.  Something like, “how many tumbles do together all the martial art practitioners in Kyoto on one given day?”.
Oh god oh god oh god. WTF. Wasn’t this the HR part of the interview??

Well, who would have thought. I had never practiced for such a thing (how?), but suddenly I was throwing up numbers all the same. Adrenaline, I guess. One factor, two, a third one, of course this is approximate but we can consider it’s good enough because of blah, … a big multiplication of it all, it’s done.

The interviewer looked at me kind of blankly. Then at the piece of paper.

“Are you sure?”, I think he asked.

I took another look at the numbers and crossed over one number and fumbled a bit with another and mumbled something but the final result stayed more or less where it was already. Which had to be a good sign, since the interviewer had started saying that just getting in the order of magnitude would be OK.  “Yes, I think that has to be something like this”.

“Oh, I wonder what would the real number be. Hehe”. Change of subject.

What… the … fuck?

You don’t even have it? What kind of dumbness is this? What use is to come up with all kind of assumptions if you are not checking the results?

How can then we know if I was lucky or not??

Yep, I remembering even getting annoyed for a moment. Are we playing job interviews or what?

I only managed to understand how wrong I was about it all when I talked to my interviewers some time afterwards. Luck? Numbers? Who cares? The important thing is the process, and if you can assure yourself and others about how the assumptions are the best than can be had.

I knew that the process was important; what I didn’t get earlier was that the process was the only important thing.

The Google interviewee who explained his assumptions to come up with an approximation of the number of gas stations in the USA took advantage of things he knew about geography and population, about his own driving and mileage per gallon, about… whatever. Probably in fact that is what stroke me as brilliant, maybe just because I rarely drive so I wouldn’t have probably had the feeling for that kind of data; seeing him conjure numbers was amazing.

And I did the same! For my assumptions I had the advantage of having lived most of my life in touristic places and the knowledge of how people flock there (and the difference between a concrete event or a seasonal thing). I knew how different martial art practitioners are more or less prone to be tumbling around (say, Aikido all the time, Karate rarely :P). I knew how an average tatami looks like, how many people can be training in it at once, how they behave - because I spent a lot of time in them.
I remember the feeling of flow while I was stringing all the numbers one after another, realising how in fact I could give solid estimates for every factor. I was pulling it off!

... in my head, at least. Because turns out that I barely explained any of the reasonings behind each number. I just wrote, wrote, crossed out something, started again, mumbled, wrote. I just wanted to be fast and get a lucky result.

So, one ends, feeling even smug about it. Fuck yeah, I aced it. The other looks blankly, not knowing what to do with a number. “…heh. Yeah, well. OK, let’s move on.”
(looking back, maybe they should have also re-stated the goals, since clearly they were more expecting a rather longish string than a number :P. I can only assume that we cross-guessed each other. Also funny that I could sense his awkwardness but didn't know what to do of it... way to waste a pretty nice hint!)

An even more interesting take is that it is not even about having the background / experience to imagine the correct numbers. I should have been able to sell him on my numbers, even if they had been pulled straight out of my ass in real time. Because if they are right that’s nice, but no one is going to notice because he didn’t have anything to compare to. Which means that this is not only about “reasoning being more important than data”, but that it’s about communication being even more important than reasoning. That might be even arguable, and probably for an engineer is a tad too much (as opposed, say, to a sales guy); but anyway for sure we all would prefer working with someone who communicates his idea (maybe to the point of unduly pulling you into it, Reality Distortion Field style) than with someone who can’t / doesn’t bother to explain what is going on in his head, where he’s the primadonna to an audience of 1.

Yes, that was a shocking thing to realize. Partly because it sounds so logical now, in hindsight.

And, of course, even the Google interview guy did it. After all, I had been able to glimpse (... or, had been shown) his brilliance. And that was it. The result being “correct” was icing on the cake, because he was already communicating it.

And now I am doing it, too. :P

One lingering question is, why was I so fixated with getting a number? First thing I though is that the style of other programming tests had put me on that frame of mind. But there’s a more insidious possibility: what comes to my mind when I think about solving a math problem is to tidily frame the final result, making it stand out from all the noise of the process. That’s what I guess we all did in exams. And I guess that’s even good presentation of the result, which should always be good. Only that in this case the result to be presented was not the final number, but… everything else. I was framing the unimportant part, and not bothering with the rest.

So, one other thing to learn is: make sure to check a couple of times at least if the question is being answered or not… another oh-so-logical tip, huh?