Unix Log Analysis

I frequently analyze logs from Linux command line. Typically this is the best way to get to the root cause of a problem you are having whether you are a developer, a sysadmin, or even a regular user. In the beginning, reviewing logs can be a daunting task, but there are a few tricks that make the process much more painless. I'll share a few of the things I've picked up with you here. These are all some common Unix tools. Information is readily available on them and the man pages can tell you all about them (just type man COMMAND in the terminal), but I'll simplify it down a bit and show how I use them day to day.

A note on piping and redirection

One thing we will be using a lot is pipes. This is when we place the | character between two commands to redirect output into input. The syntax is command 1 | command 2. The effect of this is to redirect the standard output of command 1 to the standard input of command 2. This allows us to perform consecutive processing operations on log text to generate our final readable output.

There is also redirection which uses the > character to send the standard output of a command to a file.  The key difference here is that a pipe sends a program's output to the standard input of another program, whereas redirection sends a program's output to a file.

A note on log locations

Log files are generally located in the /var/log directory unless otherwise specified in a configuration file. Take a look in this directory with the command ls /var/log and see what is available to you. As a desktop user you should at least have an Xorg log and a syslog with lots of juicy information for you to dig into.

cat

If you want to see the contents of a file without opening it in an editor, you can use the cat command. The syntax is cat [filename]. Multiple file names can be specified and they will be displayed one after another.

Now this is useful, but for a large log file this doesn't cut it. Enter less.

less

If we want to scroll through a large file, we can pipe the output of cat to less with cat [filename] | less. Piping with the | character will redirect the standard output of the cat command into the standard input of the less command. Alternatively, we can just call less on the file directly e.g. less [filename]. Less will open a navigation screen. We can scroll up and down in the file using the arrow keys or often our mouse scroll wheel. You can directly go to the beginning and end of the file by entering g or G respectively.

Less also allows you to search through the file by typing / followed by the text you want to search for. You can use n and N to jump to the next and previous instances of the search pattern.

While not necessarily for logs, a nifty trick if you want to edit a file you have open in less is that you can type v and the text file will be opened in the default system text editor.  To the uninitiated, beware that this editor will often be vi if not previously changed.  If you find yourself unable to exit vi, type :q!.  You can edit files in this way and when you exit the editor be returned back into less.To exit less type q.

grep

Grep allows us to search through a file for a pattern. The syntax is grep [pattern] [filename]. This will output all lines of the file that contain the given pattern specified. An alternative syntax to achieve the same thing would be cat [filename] | grep [pattern].

One useful option for grep is -v. It means filter out the given pattern. For example, if I want to show my NGINX access log without showing traffic from 127.0.0.1 I would use grep -v 127.0.0.1 /var/log/nginx/access.log.

Another useful option is -i. It means ignore case. For example, grep -i iNdEx /var/log/nginx/access.log will show me all lines that match index, INDEX, indeX, etc - case is not taken into account. An example of this might be if I want to find my graphical device descriptor from my Xorg log.

tim@localhost:/var/log$ grep -i ">device" Xorg.0.log
[    27.439] (**) |   |-->Device "nvidia"

We can also pipe the output of grep into less. If we take our example of filtering out 127.0.0.1, we can use grep -v 127.0.0.1 /var/log/nginx/access.log | less to navigate through our entire access log without 127.0.0.1 entries.

Another useful technique is that we can use regular expressions in our search. We do this using the -E option. An example would be if I was to show all 5XX range errors from my nginx access log I would use grep -E '5[0-9]{2}' /var/log/nginx/access.log. In this example [0-9] will match all numerical digits and {2} indicates that we want to match two digits in a row.

tail

Tail will show the end of the a text file. The syntax is tail [filename]. By default it shows the last 10 lines. This can be changed with -n. For example, tail -n 30 /var/log/syslog will show the last 30 lines of the system log.

One option I use all the time with tail is -f. This stands for follow. This will output the end of the log as it updates in real time. This is especially useful for debugging. Let's say that I have a web application and want to observe 5XX range errors in real time while I interact with my application. I might use the following: tail -f /var/log/nginx/access.log | grep -E '5[0-9]{2}'.

head

Head is like tail by for the beginning of the file. I use it much less than I use tail as it doesn't make much sense to follow the head, but it is still quite useful.

In Conclusion

These are a few of the tricks I use to look through logs. There is no real limit to how many pipes you can use and a lot of the power to this strategy lies in piping the output of these commands into each other. Both grep and searching within less allow for the usage of regular expressions to add variability to your searches.

Often when people are new to troubleshooting and debugging using logs, they can be a bit overwhelming. Hopefully this helps to make them a bit more approachable.

Compiling Xnp2 PC98 Emulator on Ubuntu Linux

I've taken an interest recently in early 90s Japanese computers and their games, especially the PC98.  Unfortunately most PC98 emulation is focused around Windows and much of the documentation is in Japanese.  Xnp2 is a Unix port of the Neko Project 2 emulator which is only Windows compatible.  I was having trouble getting Xnp2 to compile under Ubuntu 20.04, and for a while was just using Neko in a Windows VM.

After a great deal of scouring the web and looking through a few non English language websites, I was able to compile it from the git repository in a fairly straight forward manner. I've decided to document this succinctly here in case others encounter the same difficulty.

Without further ado, the following commands got Xnp2 compiled and installed for me:

$ sudo apt install build-essential nasm libsdl1.2-dev libsdl-mixer1.2-dev libgtk2.0-dev libxxf86vm-dev
$ git clone https://github.com/nonakap/xnp2
$ cd xnp2/x11
$ ./autogen.sh --enable-ia32
$ ./configure --enable-ia32
$ make
$ sudo make install

Please note that this was successfully done at commit hash  9a0baa69548cfa00c9f57ba26be8fc54e8f55272.

From there you should be able to start the emulator by entering xnp2 at the console.

I found when I started the emulator that the Japanese text did not initially appear.  This can be resolved by downloading the font file here, extracting it, and then pointing the emulator to it with via File > Font in the UI menu.

Unfortunately at first glance it would seem Xnp2 is not quite as accurate as Neko, but see how your mileage goes.

Note that this compilation was done on a more recent code base than the released version 0.86 on the official Xnp2 website.  If you want to compile 0.86 on Ubuntu 20.04, you need to apply the patch files which are also on that page.

 

For some great general information about the PC98, how to configure and use the emulator, and some of the games available, I highly recommend this link.

Also, this is a really great article if you want to generally learn about Japanese retro computers.  There is a fascinating history there with many important developers (Enix, Hideo Kojima, etc) having their start on these machines and many awesome games you've never heard of.

ATT Netopia Cayman 3347w Losing Connection

A local martial arts studio I handle IT for has a routine problem they encounter. Every once in a blue moon, their internet shuts off. It appears as though the router resets itself, and the wireless turns off. They have a Netopia Cayman 3347w router, and ATT DSL service.  Finding a solution for this one originally was difficult, and I'm sure I am not the only person to ever encounter this problem, so I'll detail out the steps taken here for you.

The first thing you have to do if you are on wireless is hard wire the computer to the router with an ethernet cable to get yourself on the network.  From here, open up your web browser and enter 192.168.1.254 as the address.  You will be prompted to put in a username and password.  Open another tab or windows and enter any site address.  It wont connect out, but it will prompt you to enter a new password.  Do this and click submit.  Afterward, go back to your original tab.  Enter Admin as the username and the password you just created as the password.

Next, you will be prompted for your ISP login and password.  If you don't have these on file, you can call ATT tech support, provide them with some of your billing information, and they will tell you.  Enter this information and you will be taken to the next screen which will attempt to connect to your ISP.  If this works, great, you're up and running.  In our particular situation, no matter how long you let it sit there, it wont connect.  So go ahead and just click Home to skip this and take you to the main menu.

At this point on the far left side you will see an option that says Expert Mode.  Click on this.  The router will ask you to confirm that you really want to do this.  Click yes and it will take you to a new menu with many more options.  Next, click on configure, then click WAN, and then at the bottom of the page you will click ATM.  What we'll be doing now is specifying to the modem how we want it to connect to ATT.

There will be three fields under this option.  You have VPI, VCC, and VCI.  In our particular case, the VPI was set to 8 by default, however to connect to ATT it must be set to 0.  The ATM circuit configuration should be as follows:

VPI = 0
VCC = 1
VCI = 35

After you have made sure all of these are correct, click submit.  A yellow triangle with an exclamation point will appear in the top right corner.  You will be directed to click on this.  It will take you to a menu where you can select Save And Restart.  Click on this and wait a minute.  The router will reboot with your configuration changes.  When it is finished, it will ask you to login again with Admin and the password your created in the beginning.  Once you do this, you will go to the main screen and should see under where it says "Status:" a green bar that says Up.  If this is present, your internet connection is back.

We only have to do one last thing to complete this process - turn the wireless back on.  Go back into expert mode and click on LAN.  Then click Wireless and check the box that says to turn the wireless on.  Click submit, then the yellow triangle, and save and restart again.  The router will restart and you should be back up and running.

What Is An Inverter? (and how to replace it)

I recently came across a problem that I had not encountered before. A client came to me with an HP Pavilion xz295. Over the phone she had said that her laptop would boot up and then the screen would go blank. The way she described it I got the impression that it was a software problem and so I readied myself for a quick little easy job. However upon seeing the problem itself I realized that it would require a bit of research.

First I just turned the computer on just to see the problem for myself. It booted up just fine and I waited for it to go blank. Nothing. I started using the computer and it seemed to function just fine. Then, about five minutes later, it hit. The screen seemed to just turn off. At that point I realized it was a hardware problem - slightly more difficult for me. I took a second look at the screen and realized there was a faint shadow of the desktop. The computer was still functioning, and you could make out shadows of what you were doing on it. This indicated that something was wrong with the backlight of the LCD.

I looked around a bit on Google for a solution to the problem. I found that most likely the LCD inverter was broken. My first thought was along the lines of "a what?". I knew basically how CRT monitors worked, but not so much about LCDs. So of course, I did some curiosity research. LCDs apply filters to light passed through them to create color on the screen, but all that is behind the screen is some light source. In the case of laptop LCDs, it is an electroluminescent panel (ELP). The only problem with using ELPs is that they require high voltage AC power. Enter the inverter, which serves the purpose of converting the laptop's DC power to AC for the ELP.



The actual replacement of the part was pretty easy. Just unscrewed the bezel (frame around the screen) and removed it with a guitar pick. The inverter was located at the bottom, right under the screen itself. First, I checked to make sure the connection wasn't loose, and once I was sure of that I went out and got the part. It was only connected in two places, so it wasn't hard to install the new one. Then just screwed on the bezel again and viola, we had a working monitor again.

printf("Hello World");

Hello and welcome to my new blog. My name is Tim and I fix computers for a living. I figured before I started posting regular articles to this blog, I should introduce myself. I am first and foremost a computer geek. I love studying the inner workings of computers. I currently have my own computer repair business in California's east bay area and San Joaquin Valley called Tracy Computer Guru. Its just me operating it right now, but someday I'll have lots of employees.

One of the best parts of my job is learning new things. I get a lot of computers to fix that I can fix with my current knowledge, but some jobs require me to do research and make discoveries for myself. The latter are the jobs that make my life exciting, and they are the reason I am bringing this blog into existence. Sometimes I have to search very hard for this information, and I am sure it would be useful to many other IT workers and computer geeks out there. I could also use a place to record these little discoveries for myself to read in the future, lest I forget. From The Desk Of The Computer Guru is a blog to put all of this knowledge in one place and provide it to others. I hope you all enjoy.