PayPal Express Checkout Not Sending Order Emails To Guests In Magento 1.6.1

Not only do we do custom Magento themes for clients, we also maintain a few websites where we sell our own products. We recently were notified of an issue where order emails were not being sent out on one of our stores. After further investigation we came to the conclusion that order emails were definitely not being sent out, but only when the user was not logged in and using the PayPal Express checkout method (this is the method that adds a 'Check out with PayPal' button directly to the cart page, underneath the normal checkout button). What follows is a documentation of the analysis and fix of the issue.


Get Total Number Of Products From Within Any Category

To get the total number of products in a category and show it on the product list page, just add these three lines of code.

This has only been tested on Magento 1.5 but should work on previous versions, let me know in the comments if you have any issues.

In template/catalog/product/list.phtml add these two lines of code where you need the count.

$_productCollection = $this->getLoadedProductCollection();
$count = $_productCollection->getSize();
echo $count;


Temporarily Deactivate Your Magento Store With Htaccess

Here is quick snippet of code that we use all the time when upgrading or moving a Magento store. This will stop people from creating accounts, placing orders, etc... while you are in the middle of heavy database work.

Add this to your .htaccess file in your root directory and only your IP address will be able to view your website. All other visitors will get redirected to an "updating" page.

Add this to your .htaccess file (directly after RewriteEngine On):

RewriteCond %{REMOTE_HOST} !^123\.45\.67\.89
RewriteCond %{REQUEST_URI} !/updating\.html$
RewriteRule \.*$ /updating.html [R=302,L]

Don't forget to create a file called updating.html and put the message that you want visitors to see while your site is down.


Upgrade to Shows a 404 Error on the Frontend

I just upgraded a store from 1.4.2 to and when I loaded the frontend I got a 404 error. This is a multi-store magento setup. The fix is to login to the admin, go to System > Manage Stores, find the default store (which is probably disabled) and simply enable it. Voila! The frontend works now. This must be a bug in

Hope it helps someone!


How To Easily Spy On Your Competitor’s Order Volume With Magento

With a little bit of detective work, Magento can give you some really useful insight into your competitor's (or anyone else) order volume. The majority of store owners don't change their default order settings and that makes it easy to get an estimate of how many orders they are doing per day. Add in some estimates on average order value (AOV) and you can even make a rough stab at their gross sales.

Here is how it works...

Magento assigns a unique order number to every new order. By default this order number is built by the following method:


So the 20th order for store id 3 would look something like this: 300000020

Magento also assigns a different unique id to every order that can be seen in the url when viewing the order in the admin (or your customer account! <this is what we use to get data from our competitors)

Here is an example from one of our Magento stores, as you can see we have done 17,740 orders so far:

Their is a catch though. Magento generates a new quote during the checkout process. So if a customer does not fully complete the checkout process it will generate an order id but obviously the company never made the sale. This is where abandoned carts come from. Here is an example:

Order id: 30000001 (Entity id: 1) - Customer completed the order
Order id: 30000002 (Entity id: 2) - Customer did not complete the order
Order id: 30000003 (Entity id: 3) - Customer completed the order

In this example it would appear that we have 3 sales when in reality we only had 2. So this method does not give us an exact number of sales, but it does give us a max number of potential sales.

Here is how you can easily get some info on your competitors

  1. Place a small order with your competitor and create a customer account during checkout, make note of your order ID and when you view the order from My Account, make note of the order id in the URL
  2. Wait a week or two and place another order, again note the order ID and ID in the URL
  3. Subtract the second order id (the URL one) from the first and divide by the number of days you waited. That will tell you approximately how many orders per day your competitor is receiving. Take that number times estimated average order value, times 365 and you can (very) roughly estimate yearly sales.
  4. EXTRA CREDIT: If you want a more accurate number, do this every couple months and you can identify their sales trends and potentially estimate sales better. You can also do competitive research about their industry to determine what their AOV might be.

Real World Example

I am currently in the process of starting a new company and we will be using Magento as our shopping cart solution. Here is my live example of this research and what it told me...

Order #31521, Placed 5/26/2011

Order #31756, Placed 6/9/2011

If I subtract the two URL ids from each other (556-509) I get 47.

47 orders / 14 days = 3.35 orders per day!

I estimated this company's AOV to be around$50 so 3.35 orders per day * $50 AOV * 365 days in a year = $61,137 in sales per year!

As I mentioned before, Magento generates a new order id before the order is actually completed, so this number is probably higher than reality. I am also guessing at their AOV but this at least gives me a good estimate to start working from. This is a very new company and they are already doing around $50k/year in sales from one product so I am now more confident about my move into this market.

Now if you have been paying close attention, you will notice that my order IDs had a much larger increase than my order entity IDs (from the URL). My assumption is that they are using this handy extension (FREE): Set Start Order Number extension

Off topic a bit...

There is another extension that will do custom order numbers, if you are the type who likes spending money on overpriced extensions and ridiculous licensing structures, click here. But seriously, see if the Free one from Ashley Schroder works for you, the guy is an amazing Magento developer and is extremely helpful. Support him, <rant>don't support developers who will automatically upgrade your previous extensions when you install a new one and then inform you that you are violating their licensing structure and must purchase an upgrade which is, for some unknown reason, based on the number of products you have and now costs $399 when you initially purchased it for $59. Do you get an extra features? Nope. More support time? Nope. The only difference is the price. Seriously, it's absurd.</rant>

Back on topic...

Ok so the Order numbers for these two orders had a difference of 235, but as we can see from the entity IDs, they could not have had more than 47 orders. It's a simple explanation, 235 / 47 = 5. They are using an increment of 5 instead of 1. Their first order will be #3000001 and their next order will be #3000006 but the entity IDs can never lie.


If you run a Magento store, how do you stop your competitors from using this technique to find out your sales numbers?

Unfortunately that order entity id is hard to mess with. The only solution I have come up with is to manually run a bunch of orders yourself using Check/Money Order or Purchase Order and then cancel them. If anyone has a better method please let me know. This technique works extremely well on Magento sites and is therefore hard to protect yourself against.

Update from one of our developers: a very SIMPLE way to hide the order id is to extend the sales_order extension and base64 encode the order id. i actually don't understand why magento doesn't already hide this.

Now go out and do some research on your competitors! Let me know in the comments if this helps you out.


Create Custom Category Attribute – The Unofficial Step-By-Step Guide

The Problem

Magento does not allow the creation of category attributes. For whatever reason, there is no way to do this in the Admin. Sometimes it would be nice to have the ability to display additional information on a per category basis. Or even use it for calculations or what have you.

The Solution

This simple solution will help you create an attribute for a category and display it on your category view page. Pay close attention, as your Magento setup is most likely different than what I show here. Also be sure to back up your database prior to making ANY changes.

Step 1: Creating A New Attribute

magento_add_attributeFirst off, we need to create a new attribute. Simply do so in the Admin under Catalog > Attributes > Manage Attributes. I'll call my attribute category_test but you can call it anything you like. Keep in mind that it must be unique and can't contain spaces or non alphanumeric characters. Set the Scope to what you need. If you have a multi-store setup and want it to be different for each store, leave it at Store View. Otherwise set it to Website or Global depending on your needs. Set Apply To to All Product Types. Specify all the Attribute Properties but set all Frontend Properties except Allow HTML Tags on Frontend to "No".

Specify at least the Admin label under the Manage Label / Options tab. This is what will be displayed as the title in the backend as well as each store view on the frontend.


Please note that you will not be able to make any changes to this attribute through the backend after you continue with step 2. Double check to make sure your settings are correct.

Save the attribute and write down the attribute_id. You can find the attribute's id by hovering over the row in the grid and reading it from the link which is displayed in the tool tip. My attribute_id is 954. (Yours will be different, of course)

Step 2: Looking Up the Correct Attribute Entity Type ID, Attribute Set ID and Attribute Group ID In the Database

First we need to open the Magento database in a database management tool, like phpMyAdmin, and open the table eav_entity_type. Search for the row containing catalog_category in the field entity_type_code and write down the entity_type_id.

 SELECT entity_type_id FROM eav_entity_type WHERE entity_type_code = 'catalog_category'; 

In my case it is 9. Yours will most likely be different, depending on what version of Magento you started your store with.

Now open the table eav_entity_attribute and find a row that has the entity_type_id set to the entity_type_id that we just looked up.

 SELECT attribute_set_id, attribute_group_id FROM eav_entity_attribute WHERE entity_type_id = [YOUR entity_type_id here]; 

Write down the attribute_set_id and the attribute_group_id values. Look at several rows and make sure they contain the same attribute_group_id values, as you could have multiple category attribute groups in your setup.

My setup has three different attribute_group_id's: 7, 166 and 167. These group id's are actually the tabs the attributes appear under when you edit a category. If you have several ID's, pick the lowest attribute_group_id, as this is most likely the "General Information" tab. In my case this is 7.

Step 3: Changing the entity_type_id of the Attribute

Next open the table eav_attribute. Now that we know what entity_type_id our new attributes needs to have, we must change it to the entity_type_id we looked up above.

 UPDATE eav_attribute SET entity_type_id = [YOUR entity_type_id] WHERE attribute_id = [YOUR attribute_id]; 

For me, the SQL statement looks like this:

 UPDATE eav_attribute SET entity_type_id = 9 WHERE attribute_id = 954; 

Step 4: Creating an Entry In the eav_entity_attribute Table

Now we need to create a new entry in the eav_entity_attribute table. To do so, use your database management tool's editor or use the following SQL statement:

 INSERT INTO eav_entity_attribute (entity_type_id, attribute_set_id, attribute_group_id, attribute_id, sort_order) VALUES ([YOUR entity_type_id], [YOUR attribute_set_id], [YOUR attribute_group_id], [YOUR attribute_id], 0); 

Use the entity_type_id, attribute_set_id, attribute_group_id and attribute_id values you looked up earlier instead of the placeholders.

For my setup, the SQL statement looks like this:

 INSERT INTO eav_entity_attribute (entity_type_id, attribute_set_id, attribute_group_id, attribute_id, sort_order) VALUES (9, 12, 7, 954, 0); 

Step 5: Changing the Sort Order

Because we added this attribute to the database by hand, we need to set the proper sort order so it appears in the right place on the category edit page. Go to Catalog > Manage Categories and edit any category in the Admin and you will find the newly created attribute at the very first position. You can change the sort order for your attribute by changing the sort_order field in the eav_entity_attribute table. Keep in mind that the sort_order can have missing and duplicate values.

Displaying the Attribute On the Category View Page (Optional)

Now that we have our custom category attribute, we can display it on the category view page. First we check if the theme you are using already has the file /app/design/frontend/default/YOUR THEME NAME/template/catalog/category/view.phtml. If you can't find the file you must copy it to your theme's /template/catalog/category directory from /app/design/frontend/base/default/template/catalog/category/view.phtml.

Now open the file in your editor of choice and add these lines anywhere you want to display the attribute's value:

 <?php $_my_category_attribute = $_category->getData('category_test'); 	if($_my_category_attribute): echo $_my_category_attribute; endif; ?> 

Of course you need to change 'category_test' to whatever you set your attribute code to.

As allways, refresh your cache or disable it while you make changes to ensure you are actually looking at the real thing and not some phantom.

That's it. If you have any trouble, post a comment and I'll try to help you figure it all out.

Tagged as: 1 Comment

Downloading Magento Extensions – The Easy Way

The Problem

I'm sure you have had to search the Magento forums for a solution to a problem. Interestingly, a lot of problems people have go unanswered. Which raises the simple question: If you have forums but nobody answering anything, what good does it do to have forums in the first place? It seems Varien and the Magento team are more interested in the Enterprise Edition and other "big" projects than to answer user questions or even attempt to make an appearance.

Extension Keys - Are They Worthless?

I recently had download several extensions from MagentoConnect. Not wanting to download and install them right from Magento, I tried pear. Unfortunately it seems that the Magento channels are about as up-to-date as Magento team members visit the forums. Hardly at all.

There had to be an easier way to get my extensions. And there is! A little article on the Magento Wiki shows so far the BEST way to get an extension. Why they don't use this on MagentoConnect is beyond me.

Downloading Via Browser

The above mentioned article shows how to get extensions from the command line. But instead of using wget we will simply use our Browser (which wget essentially is).
Paste this URL into your browser and replace TYPE and EXTENSION_NAME-X.X.X with the type of extension and the extension name and version. That's it. So for example, for the (buggy) Amazon Payments extension, we have the extension key: magento-core/Mage_AmazonPayments. That means that the TYPE is core. To get the version, simply look at the Specs of the extension on MagentoConnect. Currently, the latest version for Amazon Payments is 1.1.3. So, our URL would be: http://connect.magentocommerce.com/core/get/Mage_AmazonPayments-1.1.3.tgz. Now you can now download the extension and ungzip and untar it. Of course, you can also do this right from SSH with wget. Which saves you downloading and uploading it to your Magento installation.

And there you have it. If you have any questions, please comment and we'll try to answer them as good as we can.


Magento 1.4.2 Multi-Website Admin Redirect Problem – A Quick Workaround


We have recently updated a Magento multi-website (multi-store) from 1.3.2 to 1.4.2 and ran into a strange bug when viewing any Order. At first we couldn't quite figure out why it was happening, but we soon realized it was a bug in the Magento Admin. We couldn't find any information on the web so far. So naturally we went looking for a solution ourselves. We set up a mutli-store Magento 1.4.2 and replicated the problem.

The Problem

In a multi-store environment, the orders are coming in from different websites. When you view the order in the Admin and click any action button in the top right corner (except "Back", of course), the action is executed, but you are being falsely redirected to a non-existing page for the store the order originated from.

Multi-Website Setup:

  • Main website: www.storeA.com (this is where we access the admin from)
  • Other website: www.storeB.com (there is no admin here. you are redirected to www.storeA.com)

The order is placed from: www.storeB.com

In the admin (www.storeA.com), clicking on the action buttons redirects to www.storeB.com/admin/admin/sales_order/view/order_id/[order_id] rather than www.storeA.com/admin/admin/sales_order/view/order_id/[order_id].

Tracking the Issue

At the moment we still don't know why this is happening in detail, but here's what we have discovered after several hours of debugging so far. After the action has been executed, you are being redirected back to the order view. This is done by this:

$this->_redirect('*/sales_order/view', array('order_id' => $order->getId()));

in /app/code/core/Mage/Adminhtml/controllers/Sale/OrderController.php. For example on line 139 in the emailAction function.
For our setup this redirects to www.storeB.com/admin/admin/sales_order/view/order_id/[order_id]. If you follow this down the rabbit hole it gets kind of confusing. But bare with me, it will hopefully make more sense in the end. The function $this->_redirect is found in the inherited class Mage_Adminhtml_Controller_Action, found in /app/code/core/Mage/Adminhtml/Controller/Action.php on line 395. The second line is where the redirect is actually set:

protected function _redirect($path, $arguments=array())
    $this->_getSession()->setIsUrlNotice($this->getFlag('', self::FLAG_IS_URLS_CHECKED));
    $this->getResponse()->setRedirect($this->getUrl($path, $arguments));
    return $this;

$this->getUrl($path, $arguments) is what we need to look for. We find it a few lines down on line 415.

public function getUrl($route='', $params=array())
    return Mage::helper('adminhtml')->getUrl($route, $params);

From here we need to go to the adminhtml helper found in /app/code/core/Mage/Adminhtml/Helper/Data.php. The function is found on line 80:

public static function getUrl($route='', $params=array())
    return Mage::getModel('adminhtml/url')->getUrl($route, $params);

And from here we go to the model adminhtml/url, actually Mage_Adminhtml_Model_Url which is found in /app/code/core/Mage/Adminhtml/Model/Url.php. The getUrl function is found on line 70. On line 78 we get the URL from the parent, which is Mage_Core_Model_Url. We find that at /app/code/core/Mage/Core/Model/Url.php. This is the file that handles ALL Url requests. DO NOT, I repeat, DO NOT edit this file. If you do, you run the risk of rendering your entire store useless.
On line 805 we find the function getUrl. On line 836 we find:

$url = $this->getRouteUrl($routePath, $routeParams);

The function for it is found on line 604. And finally, line 622 returns us with the wrong URL.

$url = $this->getBaseUrl().$this->getRoutePath($routeParams);

This is because $this->getBaseUrl() returns the base URL (e.g. www.domain.com) for the store. Following that we go to line 306, and from there to line 326 where we find that it gets the base URL for the store.

return $this->getStore()->getBaseUrl($this->getType(), $this->getSecure());

The workaround shown below will make use of this function. The trick is found on line 308, or rather 309. More on that later, though. If we follow this further, we go to /app/code/core/Mage/Core/Model/Store.php. The function we are looking for is found on line 406. Here it gets the base_url from the store configuration. The problem is, that this returns us the URL for the storeB store, which is www.storeB.com. So, this is the cause of the redirect, but not the problem. The problem is that the wrong store is set to begin with.

The Workaround

We can use the following workaround to get around the redirect, but it certainly does NOT fix the issue. Use this at your own risk.

Copy /app/code/core/Mage/Adminhtml/Controller/Action.php to /app/code/local/Mage/Adminhtml/Controller/Action.php.

Remember: NEVER edit any files in /app/code/core. You can save your changes in /app/code/local. That way you can easily find your changes and if need be, replicate them when you upgrade to a newer Magento version.

Find the getUrl function on line 415 and add the following code:

/* workaround for multi-store redirect issue in 1.4.2 */
$params['_store'] = Mage::getModel('core/store')->load(0);
/* end workaround */

Your function should now look like this:

public function getUrl($route='', $params=array())
    /* workaround for multi-store redirect issue in 1.4.2 */
    $params['_store'] = Mage::getModel('core/store')->load(0);
    /* end workaround */
    return Mage::helper('adminhtml')->getUrl($route, $params);

That's it. Save the file, refresh your cache (both Magento's and your browser's cache, just to be sure) and give it a try.

The reason this works, is because we pass along the correct store for the Admin, which always has the ID 0 (Zero). Following our trail above we find that in /app/code/core/Mage/Adminhtml/Model/Url.php on line 309 the new store is set and the correct base URL is returned.

The REAL Fix

We are still looking for the real problem and hope to find a REAL fix for it. We are aware that the above workaround seems to solve the problem, but we want to make sure the problem is fixed at the source, rather than fixing the symptom. If you would like to help track this issue, you can help us by commenting on this post. If we find a good solution, we will let you know.

Tagged as: , 3 Comments

Troubleshooting Email Sending Problems in Magento

We just upgraded from Magento 1.3 to Magento 1.4.2, switched hosting providers, and suddenly our transactional emails were not being sent out. We were using the SMTP Pro extension by Ashley Schroeder (highly recommended, btw).

The problem turned out to be with our hosting provider, Nexcess. They were blocking port 587 in their firewall. The SMTP Pro extension connects to your Google Apps account through port 587.

We figured this out using a very simple script that can be helpful in a variety of ways (this was originally found on Google Groups, I can't seem to find it again or I would be happy to give attribution):

$fp = fsockopen("smtp.gmail.com",   25,   &$errno,   &$errstr,   10);
if(!$fp) {
     echo "smtp.gmail.com 587 - $errstr ($errno) \n";
} else {
     echo "smtp.gmail.com 587 -  ok \n";

Run this php script within your root Magento directory to see if you are able to connect to smtp.gmail.com on port 587. You can easily change the port number to test access to other ports. If you get an error, contact your hosting company to have them remove the block. If you get an "Ok", time to start digging a little deeper. Good luck! Email problems can be a headache in Magento.

If you are still having issues, contact us and we may be able to help you resolve it.


Empty Attributes Showing as “No” or “N/A”? Here’s a Quick Fix!

The Problem

For some odd reason, the Magento developers decided that an empty attribute should NOT be empty, but rather "No" or "N/A", depending on its type. This is not just annoying, but in some cases can display wrong information which means confused visitors and potentially lost sales.

The Fix

Fortunately, there is a quick fix that will solve the problem until the Magento developers fix this bug. Open the file /app/design/frontend/default/[theme name]/template/catalog/product/view/attribute.phtml in an editor and find the following lines:

        <?php foreach ($_additional as $_data): ?>
				<th class="label"><?php echo $this->htmlEscape($this->__($_data['label'])) ?></th>
				<td class="data"><?php echo $_helper->productAttribute($_product, $_data['value'], $_data['code']) ?></td>
        <?php endforeach; ?>

Replace these lines with this:

        <?php foreach ($_additional as $_data): ?>
            <?php $_attribute = $_product->getResource()->getAttribute($_data['code']);
			if (!is_null($_product->getData($_attribute->getAttributeCode())) && ((string)$_attribute->getFrontend()->getValue($_product) != '')) { ?>
				<th class="label"><?php echo $this->htmlEscape($this->__($_data['label'])) ?></th>
				<td class="data"><?php echo $_helper->productAttribute($_product, $_data['value'], $_data['code']) ?></td>
			<?php } ?>
        <?php endforeach; ?>

Save the file, upload it to your server and refresh the cache. Your "empty" attributes will now be hidden!