TIBCO Silver

tibbr series: Prototyping – the art of the possible

Ok, as you are reading this blog post I want you to understand something; I consider myself at best an average developer. I think the nicest thing anyone every told me about my programming was that ‘I did ok prototypes’.

So - with that admission out in the open - let’s talk about the approach to prototypes.

When we first conceived of tibbr (way back prior to the launch of TIBCO SilverTM in June last year) we decided to follow a rapid prototype plan. Each prototype would focus on one area - essentially prove a single concept.

We broke the prototypes up into three distinct areas:

1.       Technology prototypes; looking at tools, languages and other things

2.       Architectural prototypes; how to scale etc

3.       Functional prototypes; focus on the actual user experience and functionality.

The early prototypes were of the ‘does this work’ variety. The later prototypes were ‘does this make sense’ variety. Essentially - technology/architecture then functional prototypes. This helped the teamfocus early on basic architecture questions and then later with a good baseline understanding of the technology landscape we could then focus on the real value which was prototyping the functionality.

Technology Prototypes: I always like the architecture/tech exploration phases of a project. At this stage of the project everything is possible. The first rapid prototype we did using Ruby components in Silver. We were really looking at development speed/ease against system performance tradeoffs.

What we found was that Ruby has a generally good profile.  It has a good set of libraries and gems (add-ons like ActiveRecord for example) and because we could make changes to tibbr prototype services on the fly we could cut the development time dramatically.

However it wasn’t all sunshine and roses. The first is that as the services got more ‘complex’ we needed to be very aggressive in refactoring in comparison to Java or C#. Raw processing performance of things like messages was also poorer in ruby versus java or C. But ultimately the compactness of the ruby syntax, the ability to develop the services on the fly and the richness of the ruby gems meant it was the primary component tool set.

Language/Tech Area of Responsibility Guidelines
Ruby Bulk of services. Data tier services in conjunction with ActiveRecord.  Presentation tier using Rails.
Java Pipelining services (message transformation and processing).
C# Not currently used - but likely to be leveraged in the future.
C/C++ Bulk data crunching.
Caching How to best use this to scale efficiently.
Messaging Great for pipelining and event driven responses in the system regardless of scale.

Now this looks like a fair number of moving parts. The great thing about TIBCO Silver is that there is almost no penalty to mixing multiple different technologies in a single application. This really gives us the flexibility to choose the right technology for the right problem area.

We then moved from simple tech and tool selection to logical and physical architecture.

Personally I find this part of prototyping a lot of fun; it can also be a bit frustrating. You sometimes feel that you are not making a lot of forward progress or solving problems that are kinda silly. We ran into a whole bunch of JRuby Vs Ruby issues and the usual array of OSGi classpath loading issues that slowed forward momentum and I found to be a bit of a grind. Spending a full day trying to work out why something worked in Ruby and not in JRuby and having the solution be a tiny single line code change can be a little crappy.

Anyway - architecture prototypes.

Architecture Prototypes: The primary question we were answering in the architectural prototypes was ‘what is the best level or componentization and granularity for the tibbr services’

Now this is a pretty straightforward question right? People will generally follow top down or bottom up methodologies when talking about services and components. Personally I don’t actually find either methodology useful during the prototype architecture phase.

Don’t get me wrong. I am happy to work in either paradigm. But my goals at this point in the tibbr project were much more basic:

1.       How do I scale the system

2.       How does scaling the system impact the service/component granularity

3.       How do I operationalize the architecture

For example, I could build an architectural prototype that essentially takes ALL the deployed granular services as one big application bundle and then replicate this bundle on a large number of machines using some kind of HTTP load balancer for scale.

Or I could think about the types of services that I need (user management, subject management, message pipelines etc) and break up the deployment into groups that more accurately reflect system usage. I could hypothesize that a user logging into the system needs to scale more horizontally than any other function in the system. This would in turn lead me to scale components services a user login request.

You get the idea.

Now these two approaches (and the many that I haven’t discussed) have wildly different impacts on scalability and (somewhat more importantly) on the ability to administer the system effectively.

Designing a super scalable infrastructure is worth nothing if it is almost impossible to manage and administer in an easy fashion.

So you end up with these three forces that play a huge role in determining overall system and component architecture:

1.       Componentization - how easy is it for the system to be understood and to evolve over time

2.       Scalability - how can the components scale and how easy can the system flex up and down

3.       Usability - How easy is it to administrate the system at multiple levels.

We played around a lot here. There are so many novel approaches and technologies to throw around that you really start to see how the system can come together.

I have to give a nod to SCA here. Service Component Architecture is a standard that we have used for a couple of years in our cloud and SOA products. The ability to quickly and easily break apart components, assemble them in different ways gives you an incredible freedom of expression.

For example, I had original prototyped a number of the tibbr components as separate composites. Each composite would essentially encapsulate some functionality only exposing a public interface to the rest of the world. As the number of composites and components started to grow I was easily able to create ’super’ composites which would suck in the lower level composites and expose higher levels of granularity. I could also go the other way and refine existing composites until I felt I had the right levels. All of this was achievable without having to change the underlying code itself.

The final series of prototypes - which in this case we ran in parallel - were the functional prototypes that encapsulated the application functionality AND the user experience.

Functional Prototypes: The approach here was a bit different. In tech and architecture prototypes we are exploring possible solution set. What I mean by that is that you almost start with a blank sheet of paper and then step by step fill it up with different technology/architecture decision branches.

For functional prototypes we took a completely different approach. For this stage we started by asking ourselves a simple question:

“If I was to write a press release on TIBBR - what would it say?”

Then a follow on question:

“What would the sales or marketing presentation look like?”

Now I know what you are thinking - what does a press release or a presentation have to do with a functional prototype? Well for me it is pretty simple; if I cannot distill the essence of my product into something as simple as a press release I probably don’t know what I am actually building. The same is true for presentations.

From this baseline we then began a series of discussions (arguments), designs (napkin) and user interfaces (colored pens). Each mini prototype looked at different areas of the user experience and functionality. As we got deeper into the process we had no problem going back and changing an earlier prototype.

These prototypes were ‘vignettes’ they essentially took a vertical slice of functionality and with stubbed out services or dummy data and trialed it. Did it suck? Did it work as a user would expect it to work?

For the user perspective our User Experience team was a massive help. They would help us understand human factor and usability issues that we might not have thought were big issues. Even things like color and color palettes for the application itself were prototyped.

Ok, so this looks like a whole lot of work and not much gain in terms of designing, building and releasing a product right?

Yes and no. We made a conscious decision that as we moved from prototype to design/execution that we would not re-use earlier prototypes. So we did lose out there.

However what we gained was something more valuable:

1.       We had a great technology landscape of proven technologies being used for the right problem

2.       We had a very real sense of scale and manageability

3.       We had a much deeper appreciation and understanding of the application itself.

All of this helped us to get into the next phase with a high degree of confidence. The team also had developed a great horizontal and vertical understanding of the functionality so even if one person was working on the server based components and another on the user experience they each had a shared understanding.

Stayed tuned for the next edition of this blog where we will explore the actual solution from a high level tech perspective.

  • Share/Save/Bookmark

tibbr series: How we built it

tibbr is a real-time communication service designed to deliver the right information to the right person at the right time.

We achieve this through the use of subjects. Everything in tibbr is a subject. A subject can be anything. It could be a user created subject like silver.news or it could be a person (i.e matt). People can subscribe to subjects. Subjects are multi-tiered so subscribing to news.tibco means that a message posted to news.tibco.interesting will also be delivered.

The use of subjects decouples people from content - allowing a greater degree of flexibility in what information and content/context a user wants to see.

The problem we were trying to solve was pretty simple. In any company there are many topic experts. In any company there are many discussions and conversations occurring through many different mediums be they email, wiki, content management and f2f. The challenge for any company is making sure that the right information and conversations are happening with the right people.

Here is a simple scenario. Imagine I had an idea for a new product after seeing a customer. How I start a conversation and a dialog with peers and experts on this new idea?

Traditionally you would start by getting lots of different people in a room or on a conference call. You would discuss the idea and create lines of communication and next steps. The problem with this starts right from the get go: Who are the experts and who needs to be in the meeting?

The other approach would be for me to email and/or call around people that I know within the company who I think might have an interest in this new idea. But am I certain that the people I know are the right people at the right time?

As companies grow larger the challenge of effective real-time communication and decision making gets worse. I don’t know everyone in my company. It is likely that I have pockets of expertise where I didn’t know it. Worse still - we are all overloaded by information, so often times important content and context is lost in the noise.

It is a situation that is rapidly spiraling out of control . So much information is being lost, mis-directed or just plain confusing that it is becoming difficult to take appropriate action at an appropriate time.

In the enterprise we have added wikis, content management systems and adopted email/meeting/concall processes to sort through the mess.

We built tibbr as an attempt to provide a unified communication platform to sort through this mess. Lets take a fresh look at the previous examples.

Having come from the customer site with this new idea I publish information (a tibbit) to tibbr. Now I have a choice. I can simply publish it with no additional information. In this case people who are subscribed to me (my followers) will receive this information.

However in this case lets say I refine it a little bit more. I publish my new idea on a couple of subjects:

Tibco.innovation

Discussion.newtech

Tibco.customer.innovation

These three subjects represent potentially a wide group of people (maybe) that are interested in new ideas. Now - I don’t know who is subscribed to these subjects or who is actively interested in this. I also don’t know how the user is actually getting the information delivered.  But this is the point. I have decoupled the communication vehicle from the discussion allowing a potentially larger group of people to participate.

Now lets say (for sake of argument) that there are a couple of people subscribed to one or more of these subjects that takes an interest in the idea. In a traditional world this would lead to point-point emails and side meetings etc. In tibbr we keep the context by allowing you to essentially start a conversation around the post.

People subscribed to those original subjects will also receive the follow ups and discussions.

Now the flaw in what I have laid out is that this could lead to a huge volume of traffic. One of the early contentions with most communication vehicles is that the volume of unsolicited traffic can get in the way of getting things done.

There are two points to consider:

1.       A user subscribed to a subject voluntarily because they are interested.

2.       How often and in what format messages are delivered to the end-user is configurable.

When you subscribe to a subject you have the tibbr website which keeps an active log of all your tibbr communication. However you can configure the way that information gets delivered to you. This configurability allows a user to decide how and how often to receive information.

Lets say that the CTO was subscribed to tibco.innovation. The CTO would configure how that information was delivered. He or she could decide that innovation is paramount and for new tibbits to be sent directly to the primary communication device (say SMS to a cell phone).

Alternatively the CTO may prefer to get a daily digest of interesting innovations delivered to an email address so as to read them at night - perhaps giving a broader perspective.

So lets take another look at the differences:

Old Way tibbr Way
Send the idea via a specific communication medium (email/webex etc) to a set of known participants. Decouple the content from delivery. Allow experts to choose what subjects they are interested in.
Content owner dictates how the communication occurs (I set up the meeting, I sent an email) Subscribers choice how and how often information is delivered to them.
Initial content or idea fragments into multiple separate conversations and meetings making it visibility challenging without a lot of overhead Easy to keep the conversation in one place but allowing users to communicate through whatever medium is easiest for them.

We really go back to the central concept. tibbr is about delivering the right information to the right person at the right time. At a more personal level it actually is ‘Deliver information to me that I might be interested in but do it in the way that I prefer at the rate I prefer it’

Stayed tuned for the next edition of this blog where we will talk about how we came to realize tibbr from a technology point of view.

  • Share/Save/Bookmark

tibbr– soon the information will come looking for you

tibbr

After months of hectic activity, we finally launched tibbr yesterday. tibbr is a social networking tool for the enterprise that couples the usability aspects of social micro-blogging sites with a unique subject-based addressing approach to deliver relevant information in real time to business users.

I am excited about tibbr for a couple of reasons – first, this represents a natural extension to what we do today in the enterprise, getting applications to talk to each other; we now make relevant events available to business users in real time to help them make the right decisions; second – this showcases an end-to-end composite application that has been built and delivered on the TIBCO Silver™ platform.  Based on the initial response, it does seem like the press and analyst communities are quite excited about tibbr as well.  “TIBCO takes after twitter” and “TIBCO’s tibbr aims to be enterprise twitter” are interesting reads. tibbr is generating quite a few tweets in the Tweetesphere as well. I expect more in-depth coverage in the next few weeks.

In my blog post a few weeks back I did allude to this – getting the right information to the right person at the right time.  The subject-based addressing mechanism allows users to subscribe to relevant information irrespective of whether it is a person or an enterprise application that is generating this event.  In the enterprise it is not just about people, but rather contextual information and expertise about relevant subjects. A subject can represent a user, an application, or a process relevant to a particular business user in an organization.  tibbr filters out all the static and unwanted information clutter by keeping the focus on subjects.  Want to know if the purchase order relevant to your project was approved or if the component has gone through the appropriate QA cycle or if the legal contract been signed on the deal?  Simply subscribe to that subject feed.

Excited yet? Need to know more about some of the internals of tibbr? Make sure to register to test drive tibbr. Also make sure visit the Silver blogs next week. Matt Quinn (SVP, Products and Technology at TIBCO) who conceptualized and leads the design and development effort on tibbr will provide a “behind the scenes” narrative on how we actually went about conceptualizing, building tibbr and scaling tibbr via the TIBCO Silver elastic platform.

  • Share/Save/Bookmark

Staying relevant in this new 140 character world

Three things came to my mind as I was walking out of the Read Write Web (Un) Conference last week .

•    Unconferences are here to stay: I have to admit I was a skeptic going to a conference that had no set agenda or speakers, but it quickly became evident that this was not about the speakers or the agenda; it was about the participants and what they wanted to hear and discuss.  The agenda was decided by the participants at the start of the day. The agenda had different tracks with hour-long discussion topics in each track. These well-moderated discussions turned out to be more open, collaborative, thought-provoking and, hence, more informative and effective. Every participant seemed to get something out of it.  I am not quite sure how this will work with thousands of participants, but it can very effective for smaller groups.

•    Leveraging events effectively in real time: The conference was all about real time, and it was interesting to hear others echo my observations around enterprise adoption of real-time event processing. Enterprises are starting to reap the benefits of real-time events by being able to correlate events to make the right decision. Enterprises who have integrated their applications via an event bus can now analyze these events in real time to offer better services to their customers. For instance, when a customer service representative (CSR) receives a call from an irate customer to cancel a service, the CSR can offer the right incentive based on the customer’s usage patterns. We are also starting to see enterprises reaching out to social media sites (like Twitter, Facebook) and analyzing these feeds in real time to offer enhanced services to their customers. With technologies like Cloud Computing that accelerate the delivery of applications, there is probably going to be further surge of events that needs to be correlated to make useful business decisions.

•    Context is king: If you are like me, you receive hundreds of emails a day; you are part of many different social networks that include a large number of Twitter followers and Facebook friends. You wake up and turn on your iPhone or Blackberry (or your favorite PDA) to check out the top 10 tech blog sites and the aggregated news and stock alerts. This is not just information overload; this is literally an implosion of information that has a very short life span. Content was king a few years back, but with so many aggregated sources of information, it’s not just content but context that is critical in this new world. “Getting the right information to the right person at the right time and nothing else” is going to be the winning mantra of the future.

  • Share/Save/Bookmark

Gartner reports “Tibco’s Silver Cloud-Computing Platform Shows Potential”

TIBCO Silver was covered in Gartner’s report “Tibco’s Silver Cloud-Computing Platform Shows Potential” by Jess Thompson, David Mitchell Smith, Yefim V. Natis, Massimo Pezzini, and Daryl C. Plummer, August 6, 2009.

  • Share/Save/Bookmark

Amazon Web Services(AWS) Evangelist on TIBCO Silver™

Mike Culver, Web Services Evangelist at Amazon Web Services(AWS), shares his thoughts on TIBCO Silver YouTube Preview Image

  • Share/Save/Bookmark

TIBCO Silver “First Take” - from the Analysts and the Blogosphere

It’s been about two months since we announced TIBCO Silver, and the overall response has been overwhelmingly positive. We are very pleased with the positive reaction and excitement that Silver has generated among our customers, partners, press and the analyst community. There have been quite a few articles by the press and blogs by various independent thinkers on Silver. Here is a quick synopsis of some of the analyst articles and blog posts.

On the analyst front - Gartner published its research note on TIBCO Silver (Aug 6, 2009) titled “TIBCO’s Silver Cloud-Computing Platform Shows Potential.” Authored by some of the leading Gartner analysts, the report gives us positive feedback overall on our positioning and calls out our cloud platform strategy as “genuine”(not just another AMI) and describes the offering as unique and ahead of the curve compared to the other cloud platforms. MWD published its first take on Silver titled “TIBCO gets on to the Cloud with Silver,” noting that TIBCO is the first vendor offering an enterprise class cloud platform. Several other analyst firms, including Forrester, have given us their thumbs up on Silver.

On the independent blog front - “ The next generation of cloud-development platforms” is a very interesting blog post by James Urquhart. His depiction of Silver’s three-element architecture is right on the mark, although I would like to point out that the only component that runs in our datacenter is the initial provisioning server. Once the account is provisioned, Silver and all its components run entirely on the customer’s virtual machine instances. Other interesting blog posts are “TIBCO Silver: RAD and governance for enterprise clouds” by Dennis Howlett , “TIBCO Silver – Amazon for dummies” by Maureen O’Gara, TIBCO takes PaaS plus integration capabilities to global enterprises via their choice of clouds” by Dana Gardner. These blogs explain Silver quite well and are all good reads.

We also published a couple of articles highlighting the challenges that enterprise customer have in adopting cloud computing - Enterprise Ready Rapid Delivery Platform for the Cloud and Cloud Governance.

The Silver Beta Program is in full swing. Since the beta program was kick-started on July 20th, more than 200 customers have registered. In my next blog post I will detail the interesting set of cloud use-case patterns that have come to the fore. Btw, if you haven’t done so already, please sign up for the TIBCO Silver Beta program.

  • Share/Save/Bookmark

What Makes TIBCO Silver™ Different?

Here are some hints at what makes TIBCO Silver™ different:

YouTube Preview Image
  • Share/Save/Bookmark

The Benefits of TIBCO Silver™

Here are some more details on the benefits:

YouTube Preview Image
  • Share/Save/Bookmark

TIBCO Silver™ Features & Functions

Here are a few more details regarding TIBCO Silver™ and what the features and functions are:

YouTube Preview Image
  • Share/Save/Bookmark