At a time when industrial and tech giants such as Caterpillar (NYSE:CAT) and Oracle (NASDAQ:ORCL) find the path ahead challenging, it may be time to look further afield to discover companies that are young, have bright ideas and are gaining customers and making profits that their older brethren can’t match.

Of course, there are new-media freshmen such as Facebook coming soon. But in my view, social-networking companies will find that monetizing their customers could be more difficult than they’d thought.

That’s why I’ve been looking at some of the recently public companies that fit between social media and the old world of technology. These companies have the smartest engineers because they’re doing the most interesting and challenging work. And because they’re new, those smart engineers know that the rewards are greater.

Think about it: If you’re a programming genius just out of Stanford, would you rather work for a company such as Google (NASDAQ:GOOG) or Hewlett Packard (NYSE:HPQ), where the path to compensation is well worn, where in three to five years your shares may be worth twice as much as they are now? Or would you rather to go to a young ompany where in three years your shares could be worth 1,000% more?

When the hot young engineering grads head to Silicon Valley, snapping InstaGrams and swigging Monster (NASDAQ:MNST) energy drinks, they dream of landing a job at companies with explosive emerging technologies.

And in that quest, the name Splunk (NASDAQ:SPLK) has come to have a special reverence. Such an odd name, yet it’s the epitome of what engineers and marketers get excited about: It could be the Next Big Thing — the next Google or Amazon.

Here’s why: Much the way Google has become the standard by which we search and navigate the vast amount of consumer, corporate, academic and social data that comprise the Web, Splunk has quietly become the leader in searching machine data. The company’s funny name is a play on spelunking, which is the pastime of cave exploration. And machine data, in case you haven’t heard, is the next frontier.

Every time you click, every application you use, every website you visit — virtually every breath you take — generates information that engineers call “machine data.” And it’s not just data that you and I are generating from home or the office. We’re talking about data generated by millions of servers, networks and mobile devices across the globe. According to information-technology research firm Gartner (NYSE:IT), this data is expected to grow by 650% over the next five years. So managing it has become a priority.

Splunk just went public a couple of weeks ago. Its motto is “Every Company is a Data Company.” It aims to make that massive data accessible, usable and valuable. Machine data contain valuable information on security risks, user behavior, service levels, customer experiences and more. It’s as if Big Brother grew a gigantic new set of eyeballs and is staring at and recording every damn thing that every single person or machine does or will do, from now to eternity.

I hope you enjoyed your anonymity while it lasted. Splunk’s software can collect, monitor, index, and analyze this machine data, whether physical, virtual, or cloud-based, and provide definitive records of virtually everything that’s important to a business.

The number of uses for the software is almost infinite, from real-time security monitoring to managing data loads across an IT infrastructure. Users of the software can monitor just about every aspect of a company’s existence, from a single night watchman cruising ESPN on a PC to each digital step that salespeople, customers and executives take, whether it’s believed to be useful at the time or not. Then it will sift through patterns to figure out what’s meaningful and help managers either make or save money from the insights.

A great example is Splunk’s web-analytics solutions, which allow a retailer to understand online-shopping patterns, diagnose problems with an e-commerce site and track which items are being clicked on just before a user leaves the site, all in real time. What so often makes the difference between success and failure with many technology startups is the ability to persuade clients that they need the products — and retailers should find this feature incredibly useful.

Erik Swan, Rob Das, and Michael Baum co-founded Splunk in 2003, with Swan and Das continuing in executive roles. As the business grew, the co-founders felt the need to bring in someone with expertise in running a large software-management company. So in 2008 they hired Godfrey Sullivan as chief executive. Sullivan had overseen the sale of Hyperion Solutions to Oracle for $3.3 billion. Sullivan has extensive experience, having led Hyperion for several years, as well as serving in executive roles at technology firms Autodesk (NASDAQ:ADSK) and Apple (NASDAQ:AAPL).

Splunk shipped the first version of its patented software in 2006, and already it has more than 3,700 customers in 75 countries. The software is used across every sector and industry imaginable. And the company’s revenues have been growing at a phenomenal rate, from $18 million in 2009 to $121 million at the end of this last fiscal year.

Not a lot of companies are hitting that kind of stride. Splunk isn’t profitable yet, but investors will give a pass on that while the company uses its cash flow to grow by hiring more engineers, expanding its marketing and beefing up its back office.

Splunk offered shares for the first time on April 18 at $17, though the first public trade was at $32. SPLK has inched higher since then and closed today at $32.93, putting the market cap at $3.05 billion. Trading has been brisk for a new issue, with around 285,000 shares changing hands daily.

The company’s balance sheet is strong — $32 million in cash and only $2.2 million of debt. A fast-growing company in a fast-growing industry is a recipe for stellar growth, so put Splunk on your list of newcomers to watch as it moves into appealing territory for traders.