Showing posts with label programming. Show all posts
Showing posts with label programming. Show all posts


User-site-installed python packages, and PATH modification

Python makes it apparently easy to install packages. Just use pip, or any of the other more-or-less old and deprecated ways to install them, right? (heh)

The first difficulty is that maybe your system's Python needs sudo to install those packages, and you don't want (or even can't) use it.

The definitive solution is to use virtual environments, but that can feel like going too far in the "local" direction. You might just want to have something at the user level, without having the risks of using sudo, but still global for everything that the user does.

Well, turns out that PEP 370 allows you to have user-local installs of packages. You're supposed to run pip --user install whatever.

But now you have to remember to always use the --user flag! Kinda breaks the purpose of having something "global for everything that the user does".


Grand Central Dispatch for Android?

(Sometimes after a couple of hours of research, the result is a dead-end or needs further digesting.
Still, such a result might itself still be worth remembering.
This is a Quick&Dirty report of one of those cases.)

GCD is an implementation of thread pools / queueing (+ event management (sockets, files), etc) by Apple for Mac OS X / iOS. It was open-sourced, and FreeBSD got an implementation. They are practically an abstraction and simplification over pthreads (+ events, etc), and the GCD implementation includes some kernel and compiler changes (workqueues and blocks, respectively). Implementation details at It's well performing and seemingly can help simplifying a lot.

What if one wanted to use GCD in Android? Is there anything compatible enough and that could be used at the C level?


GitFlow (and friends) with remotes: avoid the busywork

There are lots of places online where one can learn about GitFlow, but it's seemingly always discussed in a local way; the details, subtle or not, about using GitFlow when you are pushing and pulling your changes through the network are never mentioned.

Now, GitFlow is a bit long on the teeth, and some of the younger and simpler alternatives do take into consideration things as Pull (or Merge) Requests; lately some even want the original GitFlow to be considered harmful. But still, there's a common basic concept in GitFlow and the alternatives, which is local feature branches. How and when exactly to merge them back into a development branch is one of the big differences, but is rarely detailed.

The goal of this post is to gather some tips on how to keep developing in the local feature branch while staying involved on a lively repository; and how to make easier the final feature merging.


Doxygen error parsing in Eclipse CDT

Eclipse (or is it the CDT?) has, by default, some Doxygen integration: if the preferences are set appropriately, Doxygen-style comments are highlighted differently, and Doxygen commands in those comments are further highlighted.

But that's about it. A particular missing feature is that errors in the comments are not highlighted in any way; and that's what made me look for a better alternative.


repo sync --force-sync overwrites your existing repository!

I couldn't find any good explanation of what does Android's repo's --force-sync do, nor why it can be needed. So I'm reporting my painful findings in repo 1.22.


An IS_DEFINED() C macro to check whether another macro is defined

Every C programmer knows to use #defines and #ifdefs to disable compilation for a piece of code. But not everyone knows that there are better ways; the problem with this strategy is that the code becomes invisible to the compiler (and "smart" indexers), which means that it stops being checked for correctness. Also, if the rest of the code changes substantially, like variable renaming, function changing their signatures, etc, the disabled part falls out of sync and will need possibly serious work when re-enabled.

There are easy improvements to the #ifdef strategy, with the inconvenience that they add boilerplate and/or waste cycles on runtime (important on embedded environments). But there's also a way to remove both the boilerplate AND the runtime checks!


Tweaking Waf for integration with Eclipse CDT

First, the way not taken: there is an official Waf extra named "" which theoretically creates the files that Eclipse expects in a project. But that extra is from 2011; both Waf and Eclipse have evolved since then, and I didn't have the time nor inclination to dig into both Waf and Eclipse internals. Also, what would be the end result of that extra: only build support, or full indexing? No docs, no idea, forget it.

So let's make it easier. We already know that Eclipse's indexing system only needs to see the compiler command lines to get an idea of what files are related, where are the includes, what are the compiler flags, etc. So what about just making Waf spit out the command lines? And turns out, it's even easy: there is another extra called "print_commands" which does just that.


Flame Graphs vs Instruments in OS X: using Intel's Performance Counters

TL;DR: as of last update for OS X 10.11 / 7.1, you can't really create meaningful CPI Flame Graphs on OS X because of bugs in Apple's tools, both GUI and command-line. There are some alternatives and workarounds; for an alternative to a CPI Flame Graph, might be good enough. I sent 6 bug reports to Apple, and 5 of them got marked as duplicates; so maybe some Instruments post-7.1 will get good enough to allow the data extraction needed for real Flame Graphs.

I have been trying to generate Brendan Gregg's Cycles-Per-Instruction Flame Graphs on OS X (10.10), mostly to check cache-related CPU stalls. However, after working on it for a good while, it's looking to me like the given method is somewhat shaky, and the same or better insights about the code can be gotten more easily; partly thanks to, … and partly in spite of


List the DTrace providers in your machine

I don't see any official way to list the DTrace providers; you can seemingly only list ALL the probes, the >300K of them (in my Mac right now), and then you have to deal with the multitude of providers instantiated multiple times for different PIDs.

So here's a small AWK script to list the unique providers, how many instances of each are there, and how many providers are attached to each PID:


Best practices for a (GNU) makefile in the XXI century

This is not really a tutorial on makefiles; there are lots of those around the web.

But most of those are very outdated, and/or follow dubious practices. You'll end up with a makefile which was OK for a make and a compiler from the 90's, maybe even around year 2000. Even the GNU make manual recommends unnecessarily complicated things, given the capabilities gcc has grown in the last decade.

So, if you know how makefiles generally work (it's enough to know what a rule, a recipe and a variable is), and want to improve yours or know of better options, the following may help.


Fred Brooks vs Dijkstra?

In the 60's, Freed Brooks led the development of the IBM/360 system and its OS. The experience made him write the famous The Mythical Man-Month.

Turns out that Dijkstra also wrote about that system in his EWD255. And everything he complained about on that system is the standard nowadays! Sounds like he considered that a lot of the complexity that should be fixed by the computer is being dumped on the programmer; that the features touted by the IBM/360’s OS are in fact partial fixes to the bad design of the computer:


My first official Linux kernel patch

Well, that took some time. Quite an exacting process… which makes you get a new perspective of how Linux is made, and what it takes to keep loose collaboration at a high quality level.

And interesting too how such a simple patch evolved. First it made me think of the bike shed

git mental model

Finally I managed to get a mental model of git that allows me to do everything without needing to google around. And this is a quick memory dump.

There are other git cheatsheets, but this is mine.


Modelo mental de git

Por fin conseguí hacerme un modelo mental de git que me permite hacer de todo sin necesidad de google ni esquemas ni Stack Overflow ;P. Y ésta es mi chuleta / "volcado de memoria". Parece que no hay muchas cosas así en español; a ver si le sirve a alguien.

La idea era mantenerlo muy corto y al grano — como una chuleta, vaya. Asumo que sabes de qué va en general git y qué son cosas como un puntero y una lista simplemente enlazada. Si hay interés lo expandiré en algo más "tutorial".


Swapping bottom left and right panes (content and patch summary/file tree) in gitk

Gitk (as of git 2.4.2) has its bottom left and right panes the other way round to what I like: the bottom right shows the file tree or the patch summary, while the bottom left shows the contents of what is selected on the right. Isn't that kinda non-standard?

There is no official way to swap those panes, so here's a little hacky patch to do it.


Great minds think alike…

… or, "using quotes by well-known people to put forward possibly unpopular points" ;P

Sometimes I have a fully formed opinion to write about; lots of times I don't, but I have an intuition of where I want to go. And those times seeing what some of the "greats" in the field think can be a bit of a beacon.


Eclipse CDT configuration for big, makefile-based projects

It's been kinda hard to get Eclipse to work well with big CDT projects (say, the Linux kernel, or even a BuildRoot project including the Linux kernel). The documentation for Eclipse is between non-existent and unhelpful, and I only could find a post that talked specifically about preparing an Eclipse project for the Linux kernel. But that post explains nothing, and does some arbitrary things. So this is the generalization of that.

Telnet autologin with a short Expect script

Couldn't find anything concise, so here's my solution: a sweet short Expect (the CLI variety installed by default in Ubuntu) script to open a telnet session, feed in the user and password, AND continue to an interactive session.


Type punning, aliasing, unions, strict-aliasing, oh my!

Imagine that you have a struct s which sometimes needs to be volatile (because it maps to a set of hardware registers) but other times you'd rather have it non-volatile (because you are working on a bunch of such structs that have been stored, so they are no longer used to interface to the hardware and you could do without the performance penalty of volatile).

That is easy enough: you can have your variables declared volatile or not depending on the situation.

But what happens when you have a function to deal with those structs? Could it be implemented so it not only works, but does the right thing in both volatile and non-volatile structs?

The trivial way to have such a thing is by just defining 2 versions of the function (with different names of course) with the differently qualified parameters; but another possibility is to define a union type in which one member is the plain type and the other is the volatile-qualified type, and make the function parameters use that union type. Standard-wise, it should work… shouldn't it?

My first hardware bug report?

I have found what looks like a hardware bug in the AT90CAN128! I guess I could be happy in a geeky way, but the number of hours spent trying to track the problem make it a bit difficult :P.

The thing is that aborting a pending MOb in the CAN controller embedded in the AT90CAN128 can leave CONMOB in an unexpected state. I am not totally sure if this is a hardware bug… or could be Atmel's library at90CANlib_3_2 being buggy AND misleading.