The last couple months (all started when comzeradd ordered two fairphones) i am going through the Great Transition: “Moving Away from every closed source/service to free (opensource) software.”
As i have already mentioned: Moving to free/opensource applications isnt always easy. But then again, when freedom was an easy thing ?.
So the transition has a few bumps in the way.
Some of my decisions are in my twitter’s timeline and some on my wiki.
I will document EVERYTHING but I’ll do it on separated blog posts and code will be in my wiki so that the entire documentation will no be a huge mesh.
I am fortunate enough to have smarted people than me to suggest brilliant things all the time.
Their comments (twitter/blog/mail) have made my life easier and are helping me with this transition.
I would love to read your comments (just remember that i dont accept http links inside blog comments).
So let’s start !
Some time in the last week, the iscsi volume of one of our PostgreSQL went up to 98% and nagios vomited on the standby mobile.
The specific postgres database holds customer’s preferences related to our webmail.
Unfortunately the webmail is a java web app (tomcat) - custom written by some company and the source code is a spaghetti mesh. The code has also gazillion bugs, so we took a decision to migrate to an opensource php based webmail. Hopefully in the near future we will official migrate to the new webmail platform and all known problems to humanity will cease to exist.
Till that time, we have to maintain the current webmail platform and figure out how a ~500Mb database has become a nearly ~50Gb nightmare!
My knowledge on databases are not basic but to be fair i lack in experience. As a veteran standby engineer I know that I need to apply a quick & dirty patch and investigate afterworks. Also I am not afraid to ask for help! And so i did.
First thing to do: increase the volume on the storage machine. I’ve said already that we are using an iscsi partition so it’s pointless that action. In fact - no it isnt !!! The storage machine has a percentage for reserving storage for snapshots. And the increase gave us a little space to breath as the snapshots were “eating” space from the actual volume! You are probably thinking that we should resize the partition - but this is a live-production machine and we dont want a downtime on the service (umount/resize/mount).
From 98% to 93% with only one command.
Second, but most popular thing to do, was VACUUM. A colleague took that step and tried to VACUUM each table separately so not to “lock” or provoke the daemon to a crash or even worst. That gave us a 88% of free space and the time to think before we act again.
For all you people that dont know postgres, postgres doesnt delete actual data from the storage only from the database. So you need to enable autovacuum or vacuum by hand from time to time.
Of course before everything else (or even vacuum) we took a pg_dump to another partition.
But pg_dump was taking hours and hours to complete.
After further investigation, we found a table that pg_dump was getting difficult with.
Fired up a new database and tried to restore this table there.
I couldnt. There was an error of duplicates and the restoration process was failing.
Tried to figured out the duplicate entries. 20 entries! The table has only four columns and a ~ 50.000 data entries. Only 20 of them were duplicates. The amount of data in size is ~20Mb. I was looking the data/entries and removed by hand the duplicates. After that i re-index the specific table and an hour later over 20Gb were free. Down to 44% from 98% by deleted 20 entries.
At that point i was thinking that postgres is mocking me. How the hell a 20Mb table had gone over 20G ?
Now pg_dump is taking 6.5 minutes - but is still taking a long time to dump this specific table.
Tomorrow is a new day to experiment with PostgreSQL
[edit1]: Just to be fair, postgres version is 8.1
[edit2]: The VACUUM process just finished. Another 20G free !!! So in total for 20 duplicate entries a total 40G disk free! We are now at 9% from 98% of used disk.
PS: We have already discussed a lot of plans (upgrade postgres version, restore the dump to a new machine etc etc) in our department but we believe not to focus to any of them (yet) as we havent found the trigger that fired up the database from 500Mb to 50Gb. After that all plays are in hand.
Most of the people that read this blog post should already know what Two-Factor Authentication is.
For those you don’t, in short terms 2FA is when you can login to a server/site/application using two things and not only one (your password).
Something you have and something you know, like when using your bank card (something you have) with it’s pin (something you know).
There are three (3) android apps at f-droid
- Google Authenticator
- FreeOTP
- Gort
I dont want to use the google authenticator, and i havent yet tested FreeOTP, so i’ve chosen Gort to write about.
Gort is using the barada-pam package as the back-end.
Barada/Gort are based on HMAC-Based One-Time Password Algorithm and not on time sync. This is useful if the clocks are drift and the server (barada) can “catch” one-time passwords even if the counter is out-of-sync by a little.
I ‘ve built a Docker image to show barada/gort on the next Security Talk at Athens,Greece Hackerspace and below is the link on my wiki that contains the Dockerfile.
Be aware on the notes/comments on the file.
I like to read.
One of my biggest fears is that I ‘ll never be able to read all the books I’ve checked in my entire life. Even in this technological era that everything is easier and faster - the amount of free time is always less than the year before. That’s the way it is for me and it seems that i can’t do anything to fix it. So i like to keep a short book list - cause my long book list is about a million of unread books (give or take a few thousands). I understand that the previous statement is somehow an overstatement but as I’ve already mentioned in the begging of this blog post, i like to read (although i am not a very smart person).
I never had read William Gibson and I thought to start with the Neuromancer. This would be my first cyberpunk culture book EVER so I believed that i would loved it.
If you have good friends - as i do - they will find a loophole to screw with you, in everything you ‘ll ever do. And so they did! They told me to stop reading Neuromancer and start the Burning Chrome by William Gibson.
And so I did! Burning Chrome is a collection of cyberpunk - science function short stories that are written before Neuromancer and that was my first intro to cyberpunk.
After that … i was reading about Transhumanism and watched videos about it.
But the most important thing of all is the fun of knowledge !
I wish i could figured it out that when i was young, in school and not arguing all day with my teachers about everything.
So I am finally reading Neuromancer! I am at 40% on the ebook, 50 days after my first attempt to read it.
It seems that you can push a WPAD to desktops via dhcp.
My proxy is based on squid running on 8080.
I ‘ve build a WPAD file similar to the below:
wpad.dat
function FindProxyForURL(url, host)
{
return "PROXY 192.168.1.2:8080; DIRECT";
}
next thing is to publish it via a web server.
I am using thttpd for static pages/files:
how to test it:
# curl -L 192.168.1.2/wpad.dat
after that a simple entry on Dnsmasq
dhcp-option=252,"http://192.168.1.2/wpad.dat"
and restart your dnsmasq
Dont forget to do a dhcp release on your windows machine
Title: dnsmasq with custom hosts file - aka ban sites with dnsmasq
I ‘ve already said it too many times, but dnsmasq is a beautiful project for SOHO (small office/home office) environment.
I am using it as DNS caching server, DHCP server & tftpd (PXE) server and it’s amazing.
One thing i do with the dns section is that i “BAN” urls i dont like. Think something like AdBlock on firefox.
Two configuration changes:
A.
as root
wget http://winhelp2002.mvps.org/hosts.txt -O /etc/hosts.txt &&
and
B.
in /etc/dnsmasq.conf
addn-hosts=/etc/hosts.txt
You can also put the wget cmd in your crontab with the @monthly scheduler but you need to restart the dnsmasq every month!
Another amazing thing is that you can add your one entries:
echo 0.0.0.0 www.facebook.com >> /etc/hosts.txt
restart your dnsmasq service and check it:
# dig www.facebook.com @localhost +short
0.0.0.0
I have decided to expire my current PGP key:
0×5882be3def6dc21a is the long version !
in 30 days from now, on 25 Sep 2014.
You can still use it to send me encrypted msg and i will use it to digital sign emails (and other staff) till that day.
After the 25th of Sep you may assume that this key is no longer valid.
I haven’t decided yet if i want to upload or advertise my new GPG key.
“Πελάτης” - προσοχή στα quotes - αγόρασε ένα domain από εμάς.
Έπειτα και χωρίς καμία επικοινωνία με το helpdesk ή τους εμπορικούς
δήλωσε ότι εμείς κάνουμε secondary dns service για αυτόν.
Έβαλε επίσης ως MX server ένα δικό μας μηχάνημα,
το οποίο δεν είναι καν ΜΧ server.
Φυσικά παρέχουμε dns secondary υπηρεσία & backup MX service
αλλά φυσικά το κάνουμε αφού μιλήσει ο πελάτης με το helpdesk
και το εμπορικό τμήμα.
Το θράσος του υποτιθέμενου πελάτη έφτασε στο σημείο
να διαμαρτυρηθεί εντόνως γιατί κι ενώ αυτός στην ζώνη του
τα έχει περάσει σωστά !!!!!!!! εμείς δεν του παρέχουμε
καμία από αυτές τις υπηρεσίες ?
I use dd frequently. Especially when i need a backup disk image to restore it to a disk with the same geometry. Most of the case this disk image has partitions. Mounting those partitions to my system is really easy with losetup. So i gathered some basic examples here: losetup examples .
Μιας και δεν χωράει στο twitter: Η αδελφούλα μου, μόλις ξεκίνησε να βλέπει το Breaking Bad.
Οπότε σχολιάζει την γαματοσύνη του με το παρακάτω:
“Άσε ρε, όλα βγαλμένα μέσα απο τη ζωή είναι.. στη Β λυκείου ο καθηγητής της Βιολογίας έφτιαχνε παράνομα cd και τα πουλούσε μέχρι που τον πιάσανε”
αχααχαχαχχαχαχαχαχα
This post is being written only in Greek.
[ disclaimer: Το παρακάτω είναι μια φανταστική ιστορία ]
Τα πράγματα δεν φαίνονται να πηγαίνουν πολύ καλά, οργανωτικά. Επιστρέφω μετά από 10 μέρες στην δουλειά για να διαπιστώσω ότι έχει γίνει εκ νέου αναδιοργάνωση. Νέος προϊστάμενος, υποδιευθυντής, διευθυντής κλπ κλπ κλπ κλπ - έχω πια ξεχάσει/χάσει την ιεραρχία - γενικά δεν ισχύει απολύτως τίποτα από ότι ίσχυε πριν από 10 μέρες. Το βασικό πρόβλημα είναι ότι χρειαζόμαστε υπαλλήλους και τα τελευταία χρόνια έχουμε γεμίσει - σε πλήθος - περισσότερους διοικητικούς από ότι ανθρώπους που παράγουν έργο.
Κι εκτός αυτού τέλος Ιουλίου/αρχές Αυγούστου μπήκε production νέο πληροφοριακό σύστημα για τις παραγγελιοληψίες. Το μόνο πρόβλημα που έχουμε, είναι ότι δεν δουλεύει όπως πρέπει ! Κατά την μεταβίβαση/φορητότητα (και διάφορες άλλες περίεργες λέξεις) το νέο σύστημα βγάζει inactive & expired τους πελάτες !!
Μετά από mini investigation, κι θέλω να πιστεύω ευλόγως απορία μου, ρωτώ: “Ποιο το workaround ? Πότε θα επιληφθεί ? κι Αφού δεν δουλεύει γιατί δεν επιστρέφουμε στο παλιό μέχρι να διορθωθούν τα προβλήματα ?”
και κάπου εκεί ξεκινά ο παραλογισμός:
- “Εμείς παραδώσαμε το project εντός προθεσμίας”
- “Μα δεν δουλεύει !”
- “Είναι μονάχα μια λειτουργία”
- “Μα υπάρχει ροή που βγάζει τους πελάτες inactive/expired”
- “Θα διορθωθεί σε μέλλοντα χρόνο, εμείς το παραδώσαμε στην ώρα του”
- “ΜΑ ΔΕΝ ΔΟΥΛΕΥΕΙ”
Όπως φαίνεται το να παραδίδεις κάτι εντός προθεσμίας είναι κάτι που επιτυγχάνει τους στόχους της εταιρείας.
Το να δουλεύει όμως ή όχι. Το γεγονός ότι δημιουργούμε πρόβλημα σε χιλιάδες πελάτες, επίσης όχι.
Κατά τα άλλα, πλέον αυτό το POST θεωρείτε βάση κανονισμού εργασίας ως πειθαρχικό παράπτωμα μιας και “χαλάει” το όνομα της εταιρείας !
This is pretty simple to even document, but i need a reference point !
<VirtualHost 1.2.3.4:80>
ServerName example.com
Redirect permanent / https://example.com
</VirtualHost>
dont forget to create the https virtual host, something like that:
<VirtualHost 1.2.3.4:443>
ServerName example.com
ServerAdmin admin@example.com
# Logs
CustomLog logs/example.com.access.log combined
ErrorLog logs/example.com.error.log
DocumentRoot /www/examplecom
DirectoryIndex index.html
<Directory "/www/examplecom">
Order allow,deny
Allow from all
AllowOverride All
AuthType basic
AuthName "Enter At Your Own Risk"
AuthUserFile /www/htpasswd_for_examplecom
Require valid-user
</Directory>
# HSTS
Header always set Strict-Transport-Security "max-age=31536000; "
# SSL Support
SSLEngine on
SSLProtocol all -SSLv2 -SSLv3
SSLHonorCipherOrder on
SSLCipherSuite HIGH:!aNULL:!MD5
SSLCertificateFile /certs/examplecom.crt
SSLCertificateKeyFile /certs/examplecom.key
SSLCertificateChainFile /certs/class3.crt
</VirtualHost>
Just finished
The Man Who Mistook His Wife for a Hat and Other Clinical Tales by Oliver Sacks
a book about clinical stories on neuropsychology !
next book in list:
I’ve written down some simple (i hope) instructions on creating an encrypted btrfs raid1 disk !
My notes have the form of a mini howto, you can read all about them here:
Gnu has a very interesting project for changing your mac address every time you want. I have found it very useful - apparently for security reasons!
The project has the name: macchanger and i will not tell you how to installed it !
I will tell you how i am using it. Become root and type: crontab -e
add the below line
@reboot macchanger -r wlan0 Now, every time you boot up your machine the mac address on you wireless card will be a random new one !
Happy privacy
GR
Friday 13
17.00 Ο πελάτης επικοινωνεί με το helpdesk, διότι μεταβαίνει σε νέα mail πλατφόρμα και χρειάζεται “ΕΠΕΙΓΟΝ” διαθεσιμότητα μηχανικού 19.00 - 20.00
17.30 Το helpdesk επικοινωνεί με τον standby μηχανικό
19.35 Συνδέεται ο standby μηχανικός για να δει το αίτημα του πελάτη - ΔΕΝ υπάρχει κανένα αίτημα.
Sat 14
00.00 Το helpdesk επικοινωνεί με τον standby μηχανικό, ότι ο πελάτης μόλις έστειλε το αίτημα
00.30 Ενημερώνεται ο πελάτης μέσω helpdesk ότι δεν υπάρχει άμεση διαθεσιμότητα μηχανικού – διευκρινίζεται με τον πελάτη κι επικοινωνείτε σε όλους ότι θα γίνει το πρωί.
10.30 Ο πελάτης επικοινωνεί με το helpdesk για να ολοκληρωθεί το αίτημά του για να μεταβεί στην νέα mail υπηρεσία/πλατφόρμα
11.15 Ο μηχανικός ενημερώνει το helpdesk ότι ο πελάτης ΔΕΝ έχει στείλει κανένα αίτημα. Ο πελάτης αναφέρει απλά “την επιβεβαίωση dns εγγραφών”.
Today’s work : A dockerfile to build an archlinux image with sshd
You can find my notes here: Dockerfile notes
I am a very proud member of Athen’s Hackerspace.
I am enjoying the entire 3+ years time (and money) that i’ve spend at this hackerspace. Love it.
Today was a very productive day.
With a good friend of mine, are working to setup an ansible, docker, btrfs workshop !
We want to contribute back to the community and we thought that this is a great opportunity.
We are not guru or anything like that - no, we just want to share the knowledge we are getting by spending time at hackerspace. Nothing more, nothing less. Just share our feedback to all the people that have helped us till now.
So, we are working together (collaboration) by making small steps towards to build these workshop.
Today’s work: Creating a tiny compressed archlinux docker image.
My instruction set is documented here: archlinux installation for docker.
Hopefully my next blog post will be about a simple ssh docker file.
We are trying to keep simple notes so that many people can read and use them.
I am currently pseudoparallel reading (i am currently reading one other book) The Pattern On The Stone By Daniel Hillis and i am really really impressed about the simple explanation on logical gates (boolean algebra).
Hillis is using as an example The Tinkertoy computer - the mechanical computer for playing TicTacToe !
Look at this page to take a quick look: The Tinkertoy computer
BTW The first mechanical computer is the The Antikythera mechanism 100BC to predict astronomical positions and eclipses.
Truly Amazing !