What differentiates Zimbra from standard email

1) Tags: Identify emails from important emails

2) Mashups / zimblets: Integrate other apps and services such as google maps, salesforce BI within Zimbra

3) Document authoring: wiki style interface for editing documents

4) IM: Chat integrated with Zimbra, Chat history is saved in archive

5) Briefcase: Place to save email file attachments, easily searchable without digging through emails. Briefcase can be shared with other users

6) Zimbra desktop: Client version of Zimbra

7) Compatible with outlook on the client side. Supports emails, filters, contacts, calendars and tasks in outlook. Outlook syncs directly with zimbra server.

8) Supports caldev, IMAP and POP etc.

Oracle exchange messaging MTA troubleshooting

Oracle messaging exchange MTA troubleshooting:

* The approach to troubleshooting MTA issues is to track the message in mail.log_current file, enable debugging for the problematic channel (will require MTA restart) and stop/start channels to hold message in queue or view the messages already queued. Debug channel log file records the full SMTP dialogue/handshake.

* Enable message ID in logs by adding 'LOG_MESSAGE_ID=1' in option.dat file

* Channels can be manually run by job controller by running the command :

imsimta submit channel
e.g imsimta submit tcp_local

Channel logging can be enabled by adding slave_debug and master_debug channel options in imta.cnf file.

* Channels can be stopped and started to trap a message in channel queues. Messages are queued to the next channel queue directory.

* SMTP server processes are started by dispatcher. Max simultaneous connection limit is equal to MAX_PROCS * MAX_CONNS settings in dispatcher configuration file. Dispatcher should be able to create a hidden socket file in /tmp directory for successful startup.

* Looping messages are held as .HELD files.

* Postmaster address must be present on an MTA system. Message loops are detected through RECEIVED header lines in message headers. Removal of RECEIVED headers can cause failure in loop detection. Loop detection are caused by server not recognising its variant name or user forwarding error etc. Messaging software should correctly handle notification emails by adding empty envelope From: address. Message can be held due to user or domain status of hold

* SMTP protocol only supports ASCII 7-bit characters. 8-bit characters must be encoded

* Test user sieve filter with command, imsimta test -rewrite -debug -filter user@domain. Filter errors can result in message delivery failure to a particular recipient.

* Job controller is in-memory database.

* MTA makes up of job controller (which run channel jobs) and dispatcher (which creates and manages smtp server processes).

* imsimta cnbuild compiles configuration into an image.

Useful commands:
imsimta process To verify MTA processes including dispatcher and job controller
imsimta submit To manually run a channel through job controller
imsimta qm stop/start channel To start or stop a channel
imsimta cache -view To view queue cache used by job controller
imsimta cache -sync To force synchronisation of in-memory cache with on disk queues
imsimta test -rewrite To validate MTA configuration without sending emails

Symantec DMP in-depth

Effective multi-pathing driver should meet throughput performance, failover performance and scalability requirements as well as business requirements of TCO and data centre agility. DMP provides full integrated support of SCSI-3 persistent reservation and I/O fencing. It can be managed from Veritas storage foundation manager.

* Below are the type of storage array in terms of multipathing:

Active-Active:

If disk array accepts and executes I/O requests to a single LUN on both controllers. No concept of primary and secondary paths.


Active-Passive:

Array accepts and executes I/O requests on one or more ports of one controller only (primary) but is able to switch to the other controller in case of failure. A typical AP array will trigger failover (LUN trespass) for a LUN if I/O is received on secondary path

Active-passive concurrent array (A/P-C) accepts and executes I/O requests to a LUN on two or more ports of the same controller.

Explicit failover (A/PF) array: Active passive array that does a LUN trespass only when it receives model specific SCSi commands from the hosts.

LUN group failover (A/PG) array: Active passive array that has a group of LUNs that are failed over simultaneously.

Active-Active asymmetric (A/A-A) array: Comply with ALUA method in SCSI-3 standards. LUN in this array can be accessed through both controller without dramatic consequences. One of the two paths provide sub-optimal performance. No concept of primary and secondary paths. This array behaves more like AP arrays during normal operation and AA arrays during failover.

VxVM shows type field for each path for AP arrays since they have a concept of primary and secondary paths. Accessing a LUN through its secondary controller/path can result in LUN trespass or I/O failure.

General:
For single-path devices, VxVM virtualization layer directly sends I/O requests to OS SCSI driver without passing them to DMP. This is called 'fast path' access. For complex arrays, DMP requires the use of array-specific array support libraries (ASLs) possibly coupled with array policy modules (APMs). ASL and APM allows DMP to claim devices during device discovery, correlate paths to same devices, gather device attributes, identify the array and the set of commands to manage multi-paths to that array. I/O software stack has file system or raw device access at the top followed by VxVM virtualization layer, DMP path management, OS SCSI driver and OS HBA driver at the bottom. DMP maximises throughput by performing load balancing. DMP offers six I/O load balancing policies, balanced path , round robin (default for AP array), minimum queue length (default for AA array), adaptive, priority, single active path. DMP manages failover time independently of the number of LUNs. In version 5.0, DMP has a fully multi-threaded, concurrent and non-blocking core design comprising of a pool of worker threads in kernel. DMP bypass SCSI layer and sends queries directly to HBA once an I/O is not returned to it. A restore task is created every 300 seconds (dmp_restore_interval). This task can restore previously failed paths. Path analysis is done by issuing one or more SCSI inquiry command. When a DMP I/O request to SCSI driver fails within dmp_failed_io_threshold interval, DMP begins error recovery by issuing SCSI inquiry commands on the suspect path. HBA transforms data between memory and I/O bus or storage format. HBA card and SAN switch parameters may impact DMP operations. Link-down timeouts and link retry count are specific to HBA driver. SAN switch specifics are interoperatability mode, buffer credits,

Below device discovery commands are supported by DMP.

devfsadm (Solaris): Performs subsystem scan, updates the device tree and loads drivers if necessary
cfgmgr (AIX), ioscan and insf -e (HP-UX) and makedev (Linux)

Terms:

Veritas DMP node name: is the meta device name which represents multiple paths to a disk. DMP node name is generated from device name according to VxVM naming scheme.
vxdmpadm: Command line interface to VxVM DMP feature
Veritas metanode: Structures created by VxVM in /dev file system for each storage device that it detects. Each node represents a metadevice.
vxconfigd: Daemon that identifies multiple paths to a device by issuing a SCSI inquiry command to each OS detected device. Daemon also rescans the storage configuration and updates in-memory data structures to reflect changes since last scan. Builds a VxVM version of device tree. For each device, each DDL calls each installed ASL in turn until an ASL claims the device based on vendor and product identifiers.
Disk access name (DANAME):
Disk media name (DMNAME):
ASL & APM: ASL tuning or disabling unused ASLs are not required in version 5 and above. APM are dynamically loadable kernel modules invoked by vxdmp driver. APM implements functions such as I/O request handling, error handling, get path state, LUN group failover, explicit failover and failover path selection.
vxesd: Event source daemon monitors event on the system to trigger appropriate DMP config updates. Gathers fabric topology information.
dual port disk drives:
DMP subpath failover groups: Set of LUN paths that are supported by the same two HBA and controller ports.
DMP suspect path and pro-active failure handling:
SNIA HBA API provided by modern HBA drivers

Useful commands:

vxdmpadm getdmpnode dmpnodename=[]
vxdmpadm getdmpnode encosure=[]
vxdisk path (Displays information about VxVM metadevices and the paths to which they correspond)
vxddladm listsupport (ASLs and the devices that they support)
vxdmpadm iostat show all (shows the effect of load balancing policies)
vxdmpadm gettune all

Veritas detected 2540 as A/P-C array
Veritas detected 3510 as A/A array

Tunable DMP variables:

dmp_restore_policy: values check_all and check_disabled
dmp_restore_interval
dmp_daemon_count
dmp_retry_count, No of times SCSI inquiry command is resent before failing a path


Reference:
http://sfdoccentral.symantec.com/sf/5.0MP3/solaris/html/vxvm_admin/ch03s01.htm
http://www.symantec.com/connect/articles/veritas-storage-foundation-50-dynamic-multi-pathing-optimizing-availability-and-performance

Examples:

root@mail2 # /etc/vx/diag.d/vxdmpinq /dev/vx/rdmp/disk_5s2

Inquiry for /dev/vx/rdmp/disk_5s2, evpd 0x0, page code 0x0
Vendor id : SEAGATE
Product id : ST373207LSUN72G
Revision : 045A
Serial Number : 054332W230

Solaris 10 new features: SMF

* Service Management facility stores service information in a configuration repository. Replaces rc scripts. SMF can restart failed services, define and apply service dependencies. Dependency works by not running a service until the service on which it depends on is also started. SMF can start services in parallel reducing boot time. Each system and application service under SMF control is an object which can be controlled by SMF commands. SMF has better debugging and logging features than rc scripts.

* Service state can be uninitialized, disabled, offline, online, degraded, maintenance, legacy_run.

* init process is still the first process that Solaris 10 kernel starts at boot

Terms and files:
* SMF master restarter daemon (svc.startd)
* SMF configuration daemon (svc.configd) manages access to SMF repository. Repository database resides in /etc/svc/repository.db.
* Service instance: Allows multiple instances of the same service, such as multiple web services running on separate ports but sharing a common configuration
* SMF manifest: XML file that contains a complete set of properties associated with a service or service instance. Manifest files are stored in /var/svc/manifest directory. Service configuration properties are imported from the manifest to SMF repository using svccfg import command.
* Service methods: Used by restarter to interact with the service. Typically reside in /lib/svc/method directory. Methods are often shell scripts similar to traditional rc scripts. There can be a start method, stop method and optionally a refresh method.
* Service log file: Stored in /var/svc/log directory.
* Fault Management Resource Identifier (FMRI): Used by SMF administrative commands to specify the service instance to be acted upon. Contains three parts separated by colons. svc:/network/login:rlogin
* Repository: Stores service configuration information in local memory and local files. Enables persistent view of service state, enables or disables service and provides a unified interface to get or set set properties.
* Delegated restarter, when svc.startd delegates its restart capability to another restarter such as inetd for a particular service or service instance
* Milestone is a special type of SMF service that merges several service dependencies. It declares a specific state of system readiness on which other services can depend. Boot command allows to boot the system directly to one of the milestone.

Commands:
svcs -a (lists all enabled and disabled services)
svcs -x (for troubleshooting a service)
svcadm (enables, disables, restarts and refresh a service, can clear a service from maintenance mode to enable it to restart)
svccfg (displays and manipulates SMF repository)
svcprop (retrieves property values from SMF repository)
inetadm (Allows observation and configuration of network services by inetd)
svcadm -t ssh (temporarily stop ssh service until the next reboot)

LDAP technicalities

* LDAP is a protocol that implements access to a directory. It was derived from X.500 protocol. Typically, it involves an LDAP client accessing an LDAP server to search or update entries in a backend directory data store. Data store can vary from a simple text file to an enterprise level database but that is irrelevant to the LDAP protocol itself. LDIF (LDAP data interchange format) is a text file which describes the content in a LDAP server and also provides a way to add/modify content.

* There are four LDAP models, Information model (describes the entries/node than contain data, these are arranged in a directory information tree DIT and contain data as per data templates or object classes), Naming model (describes how each entry in a DIT is referenced called distinguished name (DN), it is by a unique attribute of an entry. It can also be a result of multiple attributes who together are unique for any entry. DN is made up of multiple relative DN or RDN from the node in question to the root of the tree), Functional model (defines the operations that can be performed on directory data including binding, read and write operations) and security model (describes client authentication to access LDAP server and provide credentials to prove its identify).

* Object classes are of three types, structural (each entry must have one structural object class), auxiliary (which extends an object class) and abstract (which is derived from a structural class)

* LDAP naming context determines what queries will be answered to by an LDAP server and what will be referred to another server similar to DNS operation. Normally, LDAP naming is mapped to DNS domain name with an auxiliary 'domain object' object class which has a 'dc' must-have attribute. An LDAP server with a naming context of dc=example,dc=com will not answer queries in dc=com context. Initially, naming context was defined in terms of organisational and geographical units which added complexity in uniquely identifying a particular directory.

* LDAP client authentication can be anonymous, simple (clear-text), simple with TLS and SASL authentication. SASL defines a pluggable authentication mechanism where client and server can negotiate any authentication method that they both can support such as kerberos

* www.ldapzone.com is a reference source

Terms:
Entry: A node in a directory information tree. It is an instance of one or multiple object classes and serves as a basic unit of a directory. Entry is named through a DN which may or may not be an attribute of the entry.

DN: Defines how an entry is named or referenced.

Attribute: Defines the data contained in an entry.

Attribute type: Defines what can or cannot become part of an attribute

Object class: Data template which defines the attributes that may or must be part of an object class. It also contains matching rules which represents the comparison rules for attributes. Matching and encoding rules are part of an object class. Matching rules are represented by OID, which is a unique reference.

Distributed directory: LDAP directory can be implemented in a distributed architecture where a master server delegates part of the DIT to another server. The lower server only directly answers queries in the sub-naming context and refers LDAP clients from the remaining queries to the upstream server. Master server refers clients to the lower server through referrals. Server can also support recursive operations where it follows the referrals without client even knowing it

userPassword: Attribute that defines password for a particular user entry. It can be of crypt, MD5, SHA or SSHA type. SSHA is salted authentication and is considered most secure

Safely replacing a failed mirrored SVM boot disk

Below procedure can be adopted to replace a failed mirrored boot disk under SVM control. It is assumed that disk at c1t0d0 path has failed and needs replacement.

* Verify the device is the boot disk
prtconf -vp |grep bootpath

* Verify the names of SVM meta devices (d14, d13, d11, d10 and d16 in this example) on the failed drive
metastat -p

* Detach the failed drive meta devices
metadetach d4 d14
metadetach d3 d13
metadetach d1 d11
metadetach d0 d10
metadetach d6 d16

* Delete the meta devices on the failed drive
metaclear d13 d14 d11 d10 d16

* Delete the meta state database on the failed drive
metadb -d c1t0d0s7

* Unconfigure the failed drive
cfgadm -c unconfigure c1::dsk/c1t0d0

It did not work as veritas did not allow the disk to be disabled or unconfigured. Errors below:

Dec 8 00:08:39 mail1 rcm_daemon[5440]: [ID 546518 daemon.error] rcm script es_rcm.pl: VxVM vxdmpadm ERROR V-5-1-10894 Attempt to disable path failed. Last path to the disk can not be disabled. use option -f otherwise.
Dec 8 00:08:39 mail1 pseudo: [ID 129642 kern.info] pseudo-device: fcsm0
Dec 8 00:08:39 mail1 genunix: [ID 936769 kern.info] fcsm0 is /pseudo/fcsm@0
Dec 8 00:09:03 mail1 vxdmp: [ID 917986 kern.notice] NOTICE: VxVM vxdmp V-5-0-112 disabled path 32/0x8 belonging to the dmpnode 233/0x0
Dec 8 00:09:03 mail1 vxdmp: [ID 824220 kern.notice] NOTICE: VxVM vxdmp V-5-0-111 disabled dmpnode 233/0x0

* Force unconfigure the failed boot drive
cfgadm -f -c unconfigure c1::dsk/c1t0d0

* Physically replace the failed drive with a new disk

* Configure the new drive
cfgadm -c configure c1::sd1

* Format the new drive similar to the other mirrored disk
prtvtoc /dev/rdsk/c1t1d0s2 | fmthard -s - /dev/rdsk/c1t0d0s2

* Install boot blocks on the new drive
/usr/sbin/installboot bootblk /dev/rdsk/c1t0d0s0

* Install meta state database on the new drive
metadb -afc3 c1t0d0s7

* Create the meta devices
metainit -f d14 1 1 c1t0d0s4
metainit -f d13 1 1 c1t0d0s3
metainit -f d11 1 1 c1t0d0s1
metainit -f d10 1 1 c1t0d0s0
metainit -f d16 1 1 c1t0d0s6

* Attach the meta devices to begin syncing
metattach d4 d14
metattach d3 d13
metattach d1 d11
metattach d0 d10
metattach d6 d16

* Update the meta state database with the device ID of the new disk
metadevadm -u c1t0d0

* Verify new disk status, meta database and meta device status using cfgadm -al, metadb -i and metastat commands respectively.

IT control metrics that matter

* High performance organizations are defined in 'IT Process Institute (ITPI) IT Controls Performance' study done in April 2006. These organizations are characterized by two factors:

1) Actively monitor systems for changes
2) Defined consequences for unauthorized intentional changes

* Metrics that matter in ITSM:
Mean time to repair: 80% of outages are due to a change and lot of mean time to repair gets spent in figuring out what exactly changed.

First fix rate
Incidents that get fixed in first fix attempt. Microsoft Operations Framework (MOF) study shows that high-performing IT organizations reboot servers 20 times less than average and have fewer blue screens of death.

Change success rate
Changes implemented without causing incidents, service impairment or disruption. In addition a change that didn't well as per plan is also a process exception. Variance (consistently achieving targets) is a factor in successful changes

Server to system administration ratio
High performance organizations have 1 system admin for more than 100 systems. Normally, this ratio is related with %age of time spent on unplanned work.

80/20 rule applies in this case as well. 20% of set of IT controls results in 80% of realized benefits.

Symantec Enterprise Messaging Features

* IP reputation database maintains IP addresses, number of clean and spam messages from them and is used by SMTP Traffic Shaping. SMTP traffic shaping restricts or denies bandwidth used by suspicious addresses, whereby allowing unrestricted bandwidth to known clean sources. Traffic shaping decisions are not applied until the appliance has collected 50,000 messages and gathered and recorded in the local database. For best results this feature should only be enabled on email gateway devices. although it can work downstream by analyzing 'received' headers. Symantec maintains local and global reputation databases. Traffic shaping works at TCP/IP network layers.

* Bad message handling. A malformed message can cause filter hub to fail. Symantec identifies the problematic message, an alert is sent and places it in bad message queue. Bad message queue is managed by the standard mta-control command. Administrator can then take action to deliver the bad message normally, forward to a system admin's address, delete message or view and list it.

* Antivirus and antispam updates are provided from Symantec Global Intelligence Network. Antispam engine employs over 20 antispam technologies. It is 97% effective against spam emails. Symantec also has a probe network of over 2.5 million decoy accounts.

* Symantec Brightmail Gateway is available as a VMware-certified virtual appliance.

* Symantec Brightmail Gateway prevents data loss and supports incident management. Pre-built templates and dictionaries are available.

* Symantec Brightmail Gateway supports per-domain and policy based TLS encryption.

Multipathing On Unix

Storage arrays:

Active-active: A LUN to be presented on more than one controller, normally two.

Active-passive: A LUN is only presented on one controller.

Veritas DMP: Sits above Unix SCSI driver layer. It uses SCSI inquiry command to determine multiple paths to a storage device/disk. The command is '/etc/vx/diag.d/vxdmpinq /dev/vx/rdmp/Disk_0s2'. Vertias metanodes/metadevices are created in /etc/vx directory. vxdmpadm or vxdisk path shows information about VxVM metadevices. Veritas uses 'Disk access name' or 'Disk media name' to identify disks. DMP has a modular architecture. Array specific libraries (ASL) may be required as well as array policy modules (APMs)

Device discover commands in Unix:
Solaris: devfsadm (subsystem scan, updates device tree and loads device drivers)
AIX: cfgmgr (same as solaris command)
HPUX: ioscan followed by insf -e updates device tree and loads drivers
Linux: makedev only updates device tree. Scan and drive loading is mostly done at boot time
Veritas: vxdctl enable or vxdisk scandisks use vxconfigd to re-scan storage config and update in-memory data structures to reflect changes.

VPN and NAT

VPN with home/small office routers doing NAT can work with one of the following approaches:

1) Home router supporting VPN pass-through. In this case, router keeps track of tunnels originating from LAN and maps incoming ESP traffic to the same local VPN client. In this way, it avoids doing PAT for VPN traffic as ESP does not use ports like TCP or UDP.

2) VPN client and gateway supporting NAT traversal. In this case, both client and gateway are NAT-aware and encapsulate VPN packets inside UDP for the router to perform normal PAT. Client and gateway then strip off the UDP header and process the remaining packet.

By default Mac OS X VPN uses transport mode with ESP, hence integrity checksum and encryption matches on client and server regardless of the change in IP address made by the NAT device