Binary Expressions

Background Knowledge

This case was done on a machine running OpenBSD i386 v3.7 operating system with Perl v5.8.6 built for i386-openbsd using BASH as the command prompt interrupter.

I’ll be honest I have very little knowledge of Perl so if you have anything to comment on or suggest please do so.

Problem

I tried to install a couple Perl modules and received an error message of “Out of Memory!”.

Solution

Just to clarify I have very little knowledge when it comes to Perl, so this solution may or may not work for you.

On Unix machines each user has a ulimit or “user limit” which is basically from what I understand sets the limit for each resource to ensure the system is not over loaded or abused. In this case at least for me, my issue was not having a big enough data seg size. Type at the command prompt without quotes “ulimit -a” to see a list of the set limits. I increased my data seg size from 76800 to 200000. I did this by typing at the command prompt without quotes, “ulimit -d 200000″.

You may need to play with this value for your case or other ulimits. Don’t worry about the changes as none of them are permanent in this case, at least to my knowledge. When you use the ulimit it only changes it for the user you are using and for that given session. If you require more permanent changes look into using ulimit in the login script or better yet change your system configuration settings.

Comments/Thanks

Many thanks to Scott Corley who posted an excellent blog entry entitled “perl Out of memory! (with solution)” on his blog called “Red Mercury Labs”. Without this I was getting know where fast.

If anyone can provide some further knowledge on ulimit it would be much appreciated.

Source: Red Mercury Labs - perl Out of memory! (with solution)

I have learned by using the “find”, “wc” and “awk” commands one can find the total number of lines within files of a given directory and sub-directories. Thanks to NicM on IRC server Freenode channel #OpenBSD for his solution below.

Solution
find /path/blah -type f -name ‘*.php’|while read i; do wc -l <$i; done|awk ‘BEGIN { x=0 }; { x+=$1 }; END { print x }’

Note: Shell must be bourne-compatible.

I will go through a basic setup of Sendmail as a smart host. The goal is to relay all mail outgoing from an internal OpenBSD server to a gateway that will then send out the email on the server’s behalf. No inbound email traffic is permitted to the internal OpenBSD server only the outside world sees the gateway as the mail server. I’m by no means a Sendmail expert, the steps I explain here is what I’ve learned on my own and with the assistance of dhartmei and NicM in IRC EFNet #OpenBSD channel and cannonball in IRC Freenode #sendmail channel.

Setup

• Make a copy of openbsd-localhost.mc, “cp usr/share/sendmail/cf/openbsd-localhost.mc /usr/share/sendmail/cf/openbsd-localhost-smarthost.mc”.
• Edit openbsd-localhost-smarthost.mc and add this statement without double quotes “include(`/usr/share/sendmail/m4/cf.m4′)dnl”.
• Edit openbsd-localhost-smarthost.mc and add this statement without double quotes “define(`SMART_HOST’,`input_host_here’)dnl”. Note: if an IP address is used enclose the IP address with square brackets.
• Compile configuration file without double quotes, “sudo m4 /usr/share/sendmail/cf/openbsd-localhost-smarthost.mc > /etc/mail/localhost.cf”. If you receieve “permission denied”, try this instead “sudo ksh -c ‘m4 /usr/share/sendmail/cf/openbsd-localhost-smarthost.mc > /etc/mail/localhost.cf’”.
• Shutdown Sendmail (kill). Without double quotes do “sudo kill ‘head -n1 /var/run/sendmail.pid’”.
• Start Sendmail as started previously, “sendmail -L sm-mta -C/etc/mail/localhost.cf -bd -q30m”. You can see how sendmail was started by executing without double quotes “grep sendmail /etc/rc.conf*”. In my case it returned /etc/rc.conf:sendmail_flags=”-L sm-mta -C/etc/mail/localhost.cf -bd -q30m”.

openbsd-localhost-smarthost.mc example

divert(-1)
#
# Copyright (c) 1998 Sendmail, Inc.  All rights reserved.
# Copyright (c) 1983 Eric P. Allman.  All rights reserved.
# Copyright (c) 1988, 1993
#       The Regents of the University of California.  All rights reserved.
#
# By using this file, you agree to the terms and conditions set
# forth in the LICENSE file which can be found at the top level of
# the sendmail distribution.
#
#

#
#  This configuration only runs sendmail on the localhost interface.
#  This allows mail on the local host to work without accepting
#  connections from the net at large.
#

divert(0)dnl
include(`/usr/share/sendmail/m4/cf.m4')dnl
VERSIONID(`@(#)openbsd-localhost-smarthost.mc $Revision: 1.4 $')
OSTYPE(openbsd)dnl
FEATURE(nouucp, `reject')dnl
FEATURE(`accept_unresolvable_domains')dnl
FEATURE(`no_default_msa')dnl
define(`SMART_HOST',`[10.10.8.1]')dnl
MAILER(local)dnl
MAILER(smtp)dnl
DAEMON_OPTIONS(`Family=inet, address=127.0.0.1, Name=MTA')dnl
DAEMON_OPTIONS(`Family=inet6, address=::1, Name=MTA6, M=O')dnl
DAEMON_OPTIONS(`Family=inet, address=127.0.0.1, Port=587, Name=MSA, M=E')dnl
DAEMON_OPTIONS(`Family=inet6, address=::1, Port=587, Name=MSA6, M=O, M=E')dnl
CLIENT_OPTIONS(`Family=inet6, Address=::')dnl
CLIENT_OPTIONS(`Family=inet, Address=0.0.0.0')dnl
dnl
dnl Some broken nameservers will return SERVFAIL (a temporary failure)
dnl on T_AAAA (IPv6) lookups.
define(`confBIND_OPTS', `WorkAroundBrokenAAAA')dnl

Problem #1 - “[notice] child pid 13449 exit signal Segmentation fault (11)”

On a OpenBSD v3.7 i386 system running Apache v1.3.29 (not chrooted) with PHP v5.1.4 I upgraded HTML_BBCodeParser from version 1.1 to version 1.2.2 by running at command line “pear upgrade-all”. After the upgrades were complete the web site would not load a web page anymore. I looked at my HTTPD server logs and noticed I was receiving the following error message in error_log file, “[notice] child pid 13449 exit signal Segmentation fault (11)” whenever there was a HTTP request on port 80 or 443 to the web site. I couldn’t understand what the problem could be but then decided to uninstall HTML_BBCodeParser v1.2.2 and re-install HTML_BBCodeParser v1.1 to see if the web site would still function. Indeed the web site did load and function fine. So I decided to uninstall and install each version up to v1.2.2 to see where the breaking point was. As soon as the latest version of HTML_BBCodeParser v1.2.2 is installed the web site stops functioning. I was able to determine that there was something that has changed in v1.2.2 or something I did wrong in the Filters I created and existing filters I altered.

Solution #1

I uninstalled HTML_BBCodeParser then manually removed the directory /usr/local/lib/php/HTML/HTML_BBCodeParser. Then I installed HTML_BBCodeParser v1.2.2 and copied the BBCodeParser.ini example from /usr/local/lib/php/doc/HTML_BBCodeParser/BBCodeParser/example/ to /usr/local/lib/php/HTML/BBCodeParse/. After that the web site functioned as normal again.

Problem #2 - “Warning: strpos() [function.strpos]: Empty delimiter.”

However I discovered that when using PHP 5 one has to alter the BBCodeParser.ini file otherwise you will receive the following warning message when trying to load a web page using HTML_BBCodeParser, “Warning: strpos() [function.strpos]: Empty delimiter. in /usr/local/lib/php/HTML/BBCodeParser.php on line 354″.

Solution #2

Edit your BBCodeParser.ini and alter the line for the opening tag character and closing tag character to be enclosed by double quotes around the square brackets, [ and ]. Apparently this has been an issue for awhile now, http://pear.php.net/bugs/bug.php?id=2580.

Example After Alteration - BBCodeParser.ini

[HTML_BBCodeParser]

; possible values: a comma seperated list of filters
; comma seperated list of filters to use
filters = Basic,Extended,Links,Images,Lists,Email

I recently rebuilt my sendmail config file and started noticing the following errors in my /etc/maillog.

• “No local mailer defined”
• “QueueDirectory (Q) option must be set”
• “sendmail.cf line 1: invalid argument to V line: ERSIONID(@#)openbsd”

I searched for some time in Google but was unsuccessful locating how to resolve these problems. So I went to IRC Freenode in the channel #sendmail and ask for help. Cannonball suggested I add “include(`/usr/share/sendmail/m4/cf.m4′)dnl” to my config file without quotes. I added it in on the 3rd line (excluding comments). I rebuilt my config with “m4 openbsd-localhost.mc > /etc/mail/sendmail.cf”. Then killed the service sendmail (kill -HUP PID) and restarted sendmial. All the error messages disappeared. Thanks cannonball for helping out, much appreciated.

Here’s my sendmail mc file which is /usr/share/sendmail/cf/openbsd-localhost.mc with the added include line and define statement for smart_host.

divert(-1)
#
# Copyright (c) 1998 Sendmail, Inc.  All rights reserved.
# Copyright (c) 1983 Eric P. Allman.  All rights reserved.
# Copyright (c) 1988, 1993
#       The Regents of the University of California.  All rights reserved.
#
# By using this file, you agree to the terms and conditions set
# forth in the LICENSE file which can be found at the top level of
# the sendmail distribution.
#
#  This configuration only runs sendmail on the localhost interface.
#  This allows mail on the local host to work without accepting
#  connections from the net at large.
#

divert(0)dnl
include(`/usr/share/sendmail/m4/cf.m4')dnl
VERSIONID(`@(#)openbsd-localhost-smarthost.mc $Revision: 1.4 $')
OSTYPE(openbsd)dnl
FEATURE(nouucp, `reject')dnl
FEATURE(`accept_unresolvable_domains')dnl
FEATURE(`no_default_msa')dnl
define(`SMART_HOST',`10.10.8.1')dnl
MAILER(local)dnl
MAILER(smtp)dnl
DAEMON_OPTIONS(`Family=inet, address=127.0.0.1, Name=MTA')dnl
DAEMON_OPTIONS(`Family=inet6, address=::1, Name=MTA6, M=O')dnl
DAEMON_OPTIONS(`Family=inet, address=127.0.0.1, Port=587, Name=MSA, M=E')dnl
DAEMON_OPTIONS(`Family=inet6, address=::1, Port=587, Name=MSA6, M=O, M=E')dnl
CLIENT_OPTIONS(`Family=inet6, Address=::')dnl
CLIENT_OPTIONS(`Family=inet, Address=0.0.0.0')dnl
dnl
dnl Some broken nameservers will return SERVFAIL (a temporary failure)
dnl on T_AAAA (IPv6) lookups.
define(`confBIND_OPTS', `WorkAroundBrokenAAAA')dnl

Changing the name/IP address of the server is quite simple. Using your favorite text editor editing the following files and change all reference to server name and server IP address.

• /etc/hosts
• /etc/hosts.allow
• /etc/hosts.deny
• /etc/mygate
• /etc/myname
• /etc/hostname.*
• /etc/resolv.conf
• /etc/nttpd.conf
• /var/www/conf/httpd.conf

At the command prompt run the following command to reconfigure your NIC with the new IP address and applicable netmask.

ifconfig le1 inet IP_ADDRESS_HERE netmask NETMASK_HERE

The “le1″ is the device name of the NIC. Not sure what the NIC device name is? At the console prompt type without quotes “ifconfig -a” and locate the network card device name.

Source: OpenBSD FAQ - Networking

I came into work one morning and found out that my database server running OpenBSD with MySQL was no longer running and would not boot up. With a little bit of research I determined that the server went down two days before (over the weekend) and that the file system appeared to be corrupted on two partitions. A co-worker found out that the server’s primary channel on the motherboard was damage. So switched the hard drive over to the secondary channel. This I thought was odd but appeared to be true. I looked at the server and tried to get it to boot up myself. No luck as I was receiving two similar error messages as shown below. I apologize for it not being exact I was in a hurry and panicking to get the server back up and running.

Error Messages - (approximate message)

/dev/rwd0e (/var/): incorrect block count I=36484 (6 should be 0) (corr.) 15025024 DUP I=3515543.
/dev/rwd0g (/usr/): incorrect block count I=36484 (6 should be 0) (corr.) 15025024 DUP I=3515543.
Run fsck_ffs manually.

Solution

After moving the hard drive over to the secondary channel I was prompted at boot “Enter pathname of shell or RETURN for Sh:”. Now due to the fact that my CLI was in the damanged partition (/usr/local/bin/bash) I entered at the prompt “/bin/csh” without quotes and pressed enter. Now that I was into the system I ran “fsck_ffs /dev/rwd0e” and “fsck_ffs /dev/rwd0g” without quotes multiple times. It took me roughly 4 hours to resolve the problem. There was a lot of damage but nothing major or anything that was important. The database data was not affected except for a one log file.

As for how to use fsck_ffs after you run the command I just answered each questions it prompted me for. It’s quite interactive. I do ad might that I wasn’t sure what to say on some of them and wish I had a better understand of it. If anyone knows of good material that explains the fsck_ffs better and how the whole BSD file system works, please let me know.

Thanks to users in the IRC channels for OpenBSD

Saying all this I have to say thanks to all the people on IRC EFNet and Freenode in the channel #OpenBSD. Thanks to a few comments I was able to understand how to use fsck_ffs. Prior to this event I never have worked with fsck_ffs before. Again thanks everyone!

This insecurity warning occurs due to the root user’s root path is included in the environment variable PATH. The root’s path should never contain “.” as this is a security risk. For example, if the root’s path included “.” and a user happen to create a file that was named ls but contained a command to remove the $HOME the root’s home directory would be deleted.

The solution to the matter is to edit the root’s /root/.profile and remove “.” from the PATH environment variable.

Example
Current environment variable PATH:
PATH=$HOME/bin:/bin:/sbin:/usr/bin:/usr/X11R6/bin:/usr/local/bin:/usr/local/sbin:.

Remove “:.”, which would now look like this:
PATH=$HOME/bin:/bin:/sbin:/usr/bin:/usr/X11R6/bin:/usr/local/bin:/usr/local/sbin

I have a OpenBSD 3.7 i386 server setup to use Sendmail for outgoing only. Sendmail is in default configuration with only modifications to the mail aliases (/etc/mail/aliases).

I recently started noticing that I was not receiving any emails from the server. Right away I went and looked at my /var/log/maillog to see what was going on and also checked /var/spool/mqueue and /var/spool/mqueue. I noticed two things in the maillog it referred to “DSN: Service unavailable” and “dsn=5.0.0, stat=Service unavailable” and noticed a huge amount of emails sitting in my /var/spool/clientmqueue directory.

As usual I went to Google to see if I could find an answer but was having difficulty. So I went to IRC EFNet #OpenBSD channel. With some help by NicM and amonk I was able to resolve the problem. Thanks guys. The problem lies in how you have your /etc/hosts, /etc/hosts.allow and /etc/hosts.deny. Here’s what I did.

I removed any reference to localhost.domain.com in my hosts file and then also replaced ALL:LOCAL with ALL: 127.0.0.1 in my hosts.allow. The combination of the two caused the emails to not pass through. As soon as these changes were done email started passing through. If for some reason based on your configuration of those three files (hosts/hosts.allow/hosts.deny) this does not resolve you can add sendmail: localhost in your hosts.allow file. If this doesn’t resolve the problem use the below recommend resources I found useful.

Resource: Enhanced Mail System Status Codes (rfc 1893) - Look up status code meaning (i.e. 5.0.0).
Resource: MTP Service Extension for Returning Enhanced Error Codes (rfc 2034)
Resource: Web Server Talk > Email Servers > Sendmail support > DSN: Service Unavailable

It is with great disappointment to announce that I will no longer be able to provide commercial development support from my employer of Midgard CMS. I’ve been told to terminate all development with Midgard CMS. I will however not disappear in my involvement with Midgard CMS. I plan to accelerate my plans to develop Midgard CMS on a test server at home running OpenBSD v3.7 i386 and look at the possibility of development on my web hosting provider. What does this all mean? Well unfortunately this means I will not be able to provide development time like I did in the past of 5 days a week at roughly 8 hours a day. Around September I have the possibility to continue minimal development with Midgard. At that point or later on my employer will re-evaluate Midgard CMS and see again at using it as our CMS. As my employer knows and agrees that CMS is the future and is required for the web site. Unfortunately they need results now and a new site up to support there customers and representative network. How I will continue without CMS is uncertain at this time. It may not be the best direction but it will get the job done, providing results as they want.

Saying all that, I will be around. Please feel free to email or contact me via my web site. I thank everyone for your support and help.

I have Midgard 1.6.3 freshly installed and functioning 100%, at least to my knowledge. I’m able to load all the web pages and log into Aegir. I have no error messages being logged into the MidgardLog or the HTTPD error_log. However after I import my Midgard database I get blank or empty web pages rendering in the browser. I did not see any error messages being logged into my HTTPD error_log. I turned on HTTPD debug log level (ex. LogLevel debug) and then turned on my Midgard logs (ex. MidgardLog debug /var/www/logs/midgard.log). Now in my HTTPD error_log I have “[notice] child pid 265 exit signal Segmentation fault (11)” and “Midgard: Refreshing NTLM auth request for sitegroup 0. NTLM sitegroup: -1″. See the logs below for further details. The logs will show me attempting to go to /midgard/.

I get blank/empty pages when I try to go to /midgard/, /mole/ (but page is not empty contains “< ?xml version="1.0"?>” and /example/. Anyone have any ideas how to resolve these two errors?

Here’s how I imported my Midgard database of a previous install of Midgard 1.6.3.

• I logged into MySQL, executed the SQL command “drop database midgard;”.
• Executed another SQL command “create database midgard;”.
• Logged out of MySQL.
• Then I imported the data by executing the following at the shell prompt “mysql -u username -p midgard < midgard_db.sql".

After this I just opened up my browser and tried to load some web pages from that server.

Apache error_log
Processing config directory: /usr/local/midgard-1.6.3/etc/midgard/apache/vhosts/*
Processing config file: /usr/local/midgard-1.6.3/etc/midgard/apache/vhosts/scorpius.venmarces.com_80
[notice] Initializing etag from /var/www/logs/etag-state
[notice] Apache/1.3.29 (Unix) Midgard/1.6.3/Repository/MultiLang PHP/4.3.11 mod_ssl/2.8.16 OpenSSL/0.9.7d configured — resuming normal operations
[info] Server built: unknown
[notice] Accept mutex: sysvsem (Default: sysvsem)
[debug] midgard-apache1.c(1355): [client ip-here] midgard_translate_handler(UTF-8, http://scorpius.venmarces.com:80/midgard)
[debug] select_db.h(51): [client ip-here] Midgard: select_database opened database midgard
[debug] midgard-apache1.c(192): [client ip-here] Midgard: lang set to 0
[debug] select_db.h(33): [client ip-here] Midgard: select_database kept current database midgard
[debug] midgard-apache1.c(977): [client ip-here] Midgard: Refreshing NTLM auth request for sitegroup 0. NTLM sitegroup: -1
[notice] child pid 12983 exit signal Segmentation fault (11)

Midgard Log
midgard-lib:12983:debug:query=SELECT id,style,root,lang,info&1,Length(prefix),sitegroup FROM host WHERE name=’subdomain.domainname.com’ AND port in (0,80) AND ( (0 <> 0 AND prefix = ”) OR (0 = 0 AN
D prefix=Left(’/midgard’,Length(prefix)) AND IF(8>=Length(prefix),MID(’/midgard’,1+Length(prefix),1) IN (”,’/'),0)=1 ) ) AND online <> 0 ORDER BY prefix DESC, port DESC
midgard-lib:12983:debug:query=SELECT info&2 FROM page WHERE page.id=50
midgard-lib:12983:debug:query=SELECT style FROM page WHERE page.id=50

This installation example for Unix/Linux installs dependency support for Midgard CMS. Make sure Expat, zlib (comes with OpenBSD) and iconv are installed before proceeding. I have added additional configuration for MySQL (so the correct MySQL libraries are used), Microsoft SQL Server, apxs, XML, exif, dba and mbstring. Refer to the PHP configurator for details (# ./configure –help).

# ./configure –with-mysql=/usr/local –with-mssql=/usr/local –with-apxs –with-xml –with-zlib-dir=/usr/lib –with-expat –with-config-file-path=/var/www/conf –with-iconv=/usr/local/bin/iconv -–with-exif –-with-dba –-with-mbstring
# make
# make install
# make clean
# cp php.ini-recommended /var/www/conf/php.ini

Review/edit /var/www/conf/php.ini (path may vary) file to match your requirements. To meet the requirements of Midgard CMS you will require to do the following.

Add extension=midgard.so into your php.ini file.
Add the extensions path to extension_dir. Type “php-config –extension-dir”. Place the returned path into your php.ini file.
Ensure the file_uploads=on.
Ensure the short_open_tag=on.