Instantly Search Your Source Code With OpenGrok


OpenGrok Logo

OpenGrok indexes your source code to allow for instant search over a large codebase which can be an invaluable timesaver. I finished setting this up on a server at work today, here’s how.

Pre-requisites

OpenGrok requires Tomcat 6 or 7 or Glassfish, in my case I have setup using Tomcat 7 on Ubuntu.

Download & Install

Head over to the OpenGrok website and download the binary package and then unpack this in your chosen installation directory.

Now to deploy the web application simply run the deploy command and restart Tomcat (or whichever container you are using):

You should now be able to see the web interface at http://localhost:8080/source/ assuming you are using Tomcat as I am.

Get Source Code & Index

Although any directory can be used as the source code directory I have chosen to encapsulate this inside my OpenGrok directory:

Now clone / copy in all the code which you would like indexed into this directory via whatever method you feel fit. Once this is done all that is left is to set off indexing, once complete you can search the code using the web interface:

Fixes

To alter the text shown on the web front-end edit the file /var/lib/tomcat7/webapps/source/index_body.html.

In addition I found that with lots of repos selected for search Tomcat starts to fall over due to the request URL becoming much to large with all the GET variables being included, this generated Tomcat errors of the types:

To resolve this limitation, update the file /var/lib/tomcat7/webapps/source/menu.jspf, line 76 to read:

And also updatethe Container element in /var/lib/tomcat7/conf/server.xml to include the following attribute:

And then restart Tomcat:

Cron

If you made a script for updating your source code being indexed, then for extra points you can setup cron to automatically update your source code and index periodically, make a log file for this task (with a better name than the one I chose):

Add the following line in your code updater script to give a complete update / index script:

Open the cron file for your user:

And add the below cronline uses the git downloader listed previously and runs a 2am (you could of course just chain bash commands here with && or ;; rather than using a script as I have):

Summary

Done, you should now have a insta-search tool with the index updated on a nightly basis.

Aside: This is not a blog post

Posted in Code, Linux | Tagged OpenGrok | Leave a comment

It’s Been A While


Tumbleweed

Don’t you just hate it when people just don’t put the time into their blogs? Guilty. I created this blog for a few reasons; to tie together useful information for myself (and to help others where possible) and to have a central site so I can be found / contacted, but the main reason was to work on my writing skills as I felt these had suffered in recent years of being focusing mainly on code whilst cutting my teeth at my first proper dev job.

However, as is obvious, I simply don’t have the time to update this blog on a regular basis, so going forward I will be maintaining this site and may occasionally come back to share on something I’ve been working on, but generally this blog can be considered dormant if it was not already.

New Profile Site

I have built a quick and dirty profile site now at ingmarboddington.com – a domain I’ve been sitting on for a while – just to keep something up-to-date as my central presence on the web. The code for this simple site is here, with the design having been nicked forked from this guy (thanks that guy).

Stuffs

When I first started this blog I was working for Bunches doing full stack front-end work in eCommerce, where-as now I tend to be working on projects which are much more focused on backend areas of the epic pile of software at plus.net. I am intending on spending some time back on working on a couple of websites to ensure some of these previously earned skills are not left to become too out-of-date / lost forever…

Posted in GlowingMinds.co.uk | Leave a comment

Gist Page Improvements


I noted my Gists page had gone a bit wonky last week which I assumed (wrongly) to be a caching issue. Turns out GitHub now need a User Agent passed with any request to the API, in their words “Requests with no User Agent string will be rejected. If you are hitting the API without authentication, we ask that you add some kind of identification to the UA header value. This is so we can contact you if there are problems.” (source) – I guess this makes sense from their perspective.

I’ve added the user agent in the curl request and also altered my html to be more compact for better scan-ability and updated the gist with these changes.

Posted in WordPress | Tagged GitHub API, PHP | Leave a comment

Recovering Ubuntu


So I’ve been using Ubuntu as my main machine at work for the last couple of months and have already managed to break it twice (once by accidentally removing all sudoers and once from an upgrade destroying my kde install somehow), here is one way to get into a live recovery mode when something like this happens and you cannot actually get logged in to fix things.

  • Burn Ubuntu to disk (ensure that if you are recovering a 32bit machine you burn the 32bit version and same for 64).
  • Boot from the CD
  • Launch the live CD mode (“Try Ubuntu”)
  • Hit Ctrl+Alt+F1 to get to a Unix style terminal (you can return using Ctrl+Alt+F7)
  • Enter “sudo mount /dev/sda1 /mnt” (where sda1 is your linux install – check using fdisk)
  • Enter “sudo chroot /mnt”

Now you are free to try and fix whatever it was that you broke.

Posted in Linux | Tagged Ubuntu | Leave a comment

Gists Page


I’ve added a new page to the site, a Gists page, listing simple info and links to the stuff I have on GitHub:Gists. I added this as the interface over at GitHub was starting to get to me. All I really want is quick access and full visibility of the full list on my account, rather than the more expanded paginated view given on GitHub.

The WordPress page template was pretty simple, a single API request to json_decode to output. I’ve shared the template here. To use just replace your username and drop into your theme folder and the page will become available in the admin, you may want to spend a little more time styling and upgrading the markup though :P

Posted in WordPress | Tagged GitHub API, PHP | Leave a comment

Building A Web Server With A Raspberry Pi


It’s was about time I had a Linux machine at home and with the Raspberry Pi I had a perfect opportunity to get a cheap stand alone Linux build.

I’m going to go through setting up the Pi as a local web server for a PHP / MySQL development environment with support for SSH and VNC to remove the need for an extra monitor, keyboard and mouse (or an expensive KVM), as well as an FTP server so changes made on my development clients can automatically update the files on the Pi.

The Bits

By the time my Pi arrived from RS Online it seemed to be available everywhere but at least they sent the 512Mb RAM revision 2 board. The completed server cost around £50:

Item Price
Raspberry PI Box Transparent (Clear) £5.69
Raspberry Pi Type B Single Board Computer 512Mb £30.87
Genuine HTC TC-B250 Uk Mains Charger £3.49
Transcend – Flash memory card – 32 GB – Class 10 – SDHC £15.50
Total £55.55

The Pi is connected to a network switch and power only once setup, however I used a monitor through the HDMI and a USB keyboard and mouse during setup. I found the case on Amazon and was impressed by it’s quality, snug fit and strong with vents for cooling, the item is still available at time of writing here.

Flashing The SD

Needing only simple services and wanting a simple install I went for the standard Soft-float Debian “wheezy” from the Raspberry Pi site, this was unzipped and the .img file burned using the recommended Win32 Image Writer. There is a full guide over on eLinux.org covering flashing for the Pi however this is a simple process of unzip and burn.

Initial Boot

On inital boot a simple menu with initial options is presented, here you can edit some simple system settings, enable ssh and desktop on boot (which you will want to do), set your password (which you will also want to do) as well as update the config utility itself. Having gone for a large SD I used the option to expand the disk to the full capacity too. No drivers were needed for the keyboard / mouse or ethernet connection (with an active internet connection), if you don’t go for the desktop on boot option than you can simply enter startx from the prompt to launch the LXDE GUI, to restart manually use sudo shutdown -r now. It is worth noting that some of the options from the utility will be followed with a little bit of lag (during which the cursor flashes at the bottom of the screen in most cases) before coming up with the next config screen.

Static IP

For the Pi to be useful as a server the device needs to use a static IP, this can be done with a series of ifconfig commands or more easily by rewriting the /etc/network/interfaces config file to the following config set (replace gateway IP with that of your router / set the static IP to whatever you like, this will be used to connect to the device). Editing can be done using nano or classic vi which are pre-installed

Once that is done, restart the Pi again and we will now be static.

VNC

tightvncserver is a free and easy to use VNC server service which will allow for full remote desktop. To install:

Once installed, new VNC servers can be launched and killed with:

The DISPLAY value is a simple integer that allows you to separate out and set options for multiple shared desktops, use the –help flag or man to get a full list of options. But make sure to set geometry (the resolution) bearing in mind that this will also have an interface around when viewed through a client in windowed mode. I initially set mine to full 1920×1080 then realised this was less than useful as I rarely run the client at full screen (although it looks nice / works perfectly).

For the client I used tightVNC in Windows, when connecting use the IP address and display value like a port number, e.g. 192.168.1.101:1.

When looking into getting the VNC service to launch on boot the only solutions I seemed to find were quite hacky (due to x-windows / login generally being required before the service is started), I will most likely look into this issue again in the future and post whatever solution I end up with, but I have left this for now.

SSH

Assuming SSH was enabled from the inital boot utility then you will be able to connect from clients already. Simply use ssh in Linux or Putty in Windows. Connect to the IP set earlier in the interfaces config file, using the user pi and your password.

Additionally with SSH enabled you can now use your IDE or editor of choice on client machines an upload / sync to the development Pi using SFTP, again using the static IP set on port 22 with the credentials for your account on the Pi.

Web Server

Let’s install some standard stuff using apt, use the install command for each package listed below or space delimit them into a single command. Before getting started it’s also a good idea to run an auto update using apt too.

  • apache2
  • php5
  • php-pear
  • mysql-server

Now we have a basic version of a LAMP (Linux, Apache, MySQL, PHP) stack in place with a webroot of /var/www in your file system.

Once you’ve finished installing new packages (and configured a root password for MySQL), it’s a good idea to also do a quick autoclean:

Composer

For dependency management I am going to use the web development industries latest favourite toy (and with good reason) composer. Create a directory and then from the directory in terminal use:

And as a bonus add the following line to ~/.bashrc:

Now composer can be invoked from terminal simply using composer

Virtual Host

To help make our development server more useful, let’s set up a virtual host example to allow access to different ‘sites’ both on the Pi and on other machine on the local network (e.g. my windows machine where I will be using Zend Studio).

First setup a folder in your webroot (/var/www/) for your project, then backup the content of /etc/apache2/sites-enabled, then remove the contents and create a new file with a name relevant to your project, edit this with:

Replace the hostname and directory with the correct values for your site / project, I normally tend to use .dev hostname TLDs. I am using the default log locations here which can also be changed for individual projects to give better error and audit log trails.

Additionally you will need to add the site to your hosts file, open up the file /etc/hosts/ and add the line:

Do the same on clients, replacing the IP address with the static address setup for the Pi, (in windows the hosts file is hidden away in /windows/system32/drivers/etc/hosts, note that you may need to restart your networking. You can check that the DNS entry has been setup correctly with a simple ping on the hostname which should now resolve to the correct IP address.

Git

For version control I’m going to utilise Git on client machines as well as Github. Git itself can be downloaded for most platforms from the Git downloads page, once installed (make sure to include the bundled Bash goodness) for basic setup just set your name and email address for identification (as per the GitHub setup guide):

Remember that these settings will be visible on GitHub so in some cases you may want to use a dummy email address. Now you can clone / push / pull from GitHub as well as any other public Git repositories.

Summary

With all of the above I now have a nice local dev server allowing for simple staging within a pretty standard environment, with composer setup starting new projects (or just poking around any given framework becomes a breeze see my gist for composer syntax) and with SSH / SFTP / VNC / Git support we have the basic tools we need for administration and use.

Posted in Linux | Tagged apache2, composer, Debian, git, MySQL, PHP, Raspberry Pi, SSH, VNC | Leave a comment

Git Flow


I’ve not had time to complete a full post with having started my new day job last week as well as having all the usual time killers. In the meantime I quickly wanted to mention a tech that is being introduced at work.

Git flow is a methodology for branching which employs some best practices for managing releases, features and hotfixing which was formalised by Vincent Driessen. An open source project on GitHub opens a new set of commands to make adherence much simpler.

I’m not going to explain in anymore detail as others have done this better than I ever could, below are the links if interested as well as my cheat-sheet gist.

Posted in Git | Tagged git, git-flow | Leave a comment

Oracle Certified MySQL 5 Associate


On Friday I took and passed the Oracle Certified MySQL 5 Associate exam, which covered all the MySQL 5 basics.

My preparation only really involved the use of the MySQL 5.0 Certification Study Guide (up to page 300ish) making some notes and playing with the standard mysql client in Linux to test some new / niche things out. Additionally the MySQL Manual came in handy for reviewing functions and transactions.

Not really allowed to discuss the exam questions (due to a standard NDA) but after so much reading I didn’t seem to have much difficulty, but that of course follows. MySQL is not a difficult subject in general, there is simply a lot to learn.

Posted in MySQL | Tagged MySQL, Oracle | Leave a comment

The Importance of Website Architecture – OnlineRetailingBlog.com


I have written another post on The Online Retailing Blog on the topic of “The Importance of Website Architecture“.

Posted in OnlineRetailingBlog.com | Tagged Site Architecture | Leave a comment

Useful And Free Windows Applications


Little bit of a filler / linkbait post… listed below in no particular order are a bunch of links to free applications for Windows which I have found useful in the past.

System Tools

  • CCleaner – Windows cleanup tool, features hard drive and registry cleanup tools and well as simple and fast interfaces for managing installed applications and startup programs.
  • CPU-Z – Provides detailed hardware information.
  • Virtual Box – For management of virtual machines
  • WinRar – Required for the common .rar archive format. Trial version, but can’t say I’ve ever had a problem with it.
  • 7zip – Open source archiver

Media Players & Tools

  • Ashampoo Burning Studio – DVD and CD burning / ripping copying software including imaging (watch out for crapware included with this one.
  • CDex – Music CD ripper, including ODBC support.
  • VLC – Media player, works for almost all video with a large range of codecs available.
  • Winamp – The classic media player, good for music, not so much for video.
  • GOM Player – Another media player, nicer interface than VLC but without as much support.

Editors

  • Libre Office – A complete office productivity suite with support for open document formats as well as Microsoft proprietary formats
  • Notepad++ – Like Notepad, only better. Features tabbed files and sytax highlighting.

Networking & Internet Tools

  • FileZilla – FTP Client and Server.
  • inSSIDer – Wi-Fi scanner, find a channel with low congestion.
  • PuTTy – The ubiquitous SSH and Telnet client.
  • Wireshark – Network protocol analyser (network packet sniffer).
  • XAMPP – Apache distribution containing MySQL, PHP and Perl, perhaps not the best, but def one of the quickest ways to get a development environment running.
  • VNC Viewer – A client for remote desktop connection to machines running a VNC server / X desktop.
  • Google Drive – Cloud file storage from Google with 5Gb free / paid options, nice integration with other Google products
  • DropBox – Another popular cloud storage tool, good for some extra free space / integration with the WordPress Backup To DropBox Plugin
  • Git – Distributed version control, if you work in dev and don’t use Git then get on it. There is a full Git Book available online to get you started

Web Browsers

  • Google Chrome – The best browser of the moment, especially for development.
  • Lynx – Text Browser, useful for testing accessibility.
  • IETester – For testing legacy versions of Internet Explorer
  • During development testing, the other big players in the browser space are often needed, however I could never recommend Firefox (as it’s bug hell) or Safari (because it’s Apple and also likes to break stuff like fonts) or IE (because, well it’s IE) for main use.

If there is anything I have missed / you think I should try then please do tell. Meanwhile I promise that the next post will be full of Linux goodness :P

Posted in Windows | Tagged Applications | Leave a comment