With Great Power Comes Great Responsibility

The Enterprise Data Architecture, Part 3

Every year, we face challenge after challenge regarding data security.  Whether it’s personal identity theft, corporate data hacking, ransomware or industrial espionage, it seems like for every step forward, we take two-three steps backwards.  This has a large impact not only on private consumers, but on the business world in general.

For consumers, the biggest issue being dealt with is Identity Fraud.  With the increasing amount of Identity theft & fraud, consumers are facing more and more challenges to keep their personal data safe.  More and more data is being collected from the consumer, from browsing habits to shopping trends.  Additionally, critical personal information such as credit card information, phone numbers, address, or even occasionally social security numbers are being used on various online websites.  Let’s not even start a discussion about the credit card skimmers at various retailers….

And what happens after your personal data has been stolen…?

Image Source: Ponemon Institute

 

On the business side, it appears that no one is safe.  With the recent hack of Equifax, we have seen tremendous amounts of data stolen that could be devastating to consumers.  A reported 143 million people’s data was taken, potentially including Social Security numbers.  And the worse part?  None of us even gave Equifax permission to have our data to begin with.  The list of hacks for 2017, according to ZDNet, is frightening:

  • Equifax – 143 million accounts
  • Verizon – 14+ million accounts
  • Bell Canada – 1.9 million accounts
  • Edmodo – 77 million accounts
  • Handbrake (video encoder software for Mac) – unknown number of users
  • Wonga – 270 thousand accounts
  • Wannacry ransomware – 200 thousand computers
  • Sabre – thousands of business customers
  • Virgin America
  • Cellebrite
  • Cloudfare
  • iCloud
  • TSA
  • OneLogin
  • US Air Force

The list could go on.  We live in a time where data is everywhere.  This is the reality of Big Data.  Regardless of whether it’s your personal data or  business data, it is stored somewhere, and it is potentially susceptible to to unwanted exposure.  We know that hackers are going to try their best to get to that data.  So what are the real challenges for the security of Big Data?  And what can we do to make it more secure?  According to Data Center Knowledge, here is a list of the top 9 Big Data Security challenges, and potential ways the security could be improved.

Big Data Security Challenges:

  1. Most distributed systems’ computations have only a single level of protection, which is not recommended.
  2. Non-relational databases (NoSQL) are actively evolving, making it difficult for security solutions to keep up with demand.
  3. Automated data transfer requires additional security measures, which are often not available.
  4. When a system receives a large amount of information, it should be validated to remain trustworthy and accurate; this practice doesn’t always occur, however.
  5. Unethical IT specialists practicing information mining can gather personal data without asking users for permission or notifying them.
  6. Access control encryption and connections security can become dated and inaccessible to the IT specialists who rely on it.
  7. Some organizations cannot – or do not – institute access controls to divide the level of confidentiality within the company.
  8. Recommended detailed audits are not routinely performed on Big Data due to the huge amount of information involved.
  9. Due to the size of Big Data, its origins are not consistently monitored and tracked.

How can Big Data Security be Improved?

  1. The continued expansion of the antivirus industry.
  2. Focus on application security, rather than device security.
  3. Isolate devices and servers containing critical data.
  4. Introduce real-time security information and event management.
  5. Provide reactive and proactive protection.

As stated by James Norrie, dean of the Graham School of Business at York College of Pennsylvania, “Big companies made big data happen. Now, ‘big security’ must follow, despite the costs. Regulators and legislators need to remind them through coordinated actions that they can spend it now to protect us all in advance or pay it later in big fines when they don’t. But either way, they are going to pay. Otherwise, the only ones paying will be consumers.”

 

, ,

No Comments

Taming the Beast

The Enterprise Data Architecture, Part 2

So we know that Big data is a big problem.  What can we do about it and what can we do with it?

First of all, we need to have an understanding of the content of the big data and how we process this information.  Gandomi & Haider (2015) state that “the over all process of extracting insights from big data can be broken down into five stages.  These five stages form the two main sub-processes: data management and analytics. Data management involves processes and supporting technologies to acquire and store data and to prepare and retrieve it for analysis. Analytics, on the other hand, refers to techniques used to analyze and acquire intelligence from big data.”

Image source: Gandomi & Haider, 2015

 

Once we have a better understanding of the how to process Big Data, how can we efficiently leverage the information we gain?

The growth of data metrics over time in both quantity and quality is nothing new.  Even in the 80s and 90s, articles were published regarding the exponential growth of data and the challenges caused by this growth (Press, 2013).  The challenge that has been continually fought in all those years has been to make the available data meaningful.  Big data, as we have defined it, provides many opportunities for organizations through “improved operational excellence, a better understanding of customer relations, improved risk management and new ways to drive business innovation.  The business value is clear” (Buytendijk & Oestreich, 2016).  Yet, investment in big data tools and initiatives are slow to materialize.

Organizations can incorporate big data initiatives into their existing business process components and artifacts.  This will help establish big data as part of their Enterprise Architecture Implementation Plan and incorporate these new business strategies into the future architecture.  Educating business leaders and stakeholders as to the benefits of big data initiatives will help overcome any concerns they may have (Albright, Miller & Velumani, 2016).  Back in 2013, Gartner conducted a study on Big Data which showed that many leading organizations had already adopted big data initiatives to various degrees, and were starting to use the results in both innovation initiatives as well as sources of business strategy.  Included in their results from that study, they found that “42% of them are developing new products and business models, while 23% are monetizing their information directly” (Walker, 2014).

This is where Enterprise Architecture comes into play.  EA is uniquely equipped, with it’s holistic view of strategy, business and technology, to not only create a strategic plan to address the potential big data opportunities, but to create a road map which positions organizations to improve operations, enable growth, and foster innovation.  “Organizations that leverage EA in their big data initiatives are able to identify strategic business goals and priorities, reduce risk and maximize business value” (Walker, 2014).

Image source: Walker, 2014

 

Through the various efforts of EA, new technologies and new approaches to processing, storing and analyzing Big Data, organizations are gradually finding ways to uncover valuable insights.  Obviously, leading the charge are leaders such as Facebook, LinkedIn and Amazon, but many other companies have joined the effort as well.  “From marketing campaign analysis and social graph analysis to network monitoring, fraud detection and risk modeling, there’s unquestionably a Big Data use case out there with your company’s name on it” (Wikibon. 2012).  This infographic below was put together by Wikibon to show how various business leaders have overcome their challenges and have tamed the Big Data beast.

 

 

References:

Albright, R., Miller, S. & Velumani, M. (August 2016).  Internet of Things & Big Data.  EA872, World Campus.  Penn State University.

Buytendijk, F. & Oestreich, T. (2016) Organizing for Big Data Through Better Process and Governance. (G00274498). Gartner.

Gandomi, A. & Haider, M. (2015). Beyond the Hype: Big data concepts, methods, and analytics, International Journal of Information Management, Volume 35, Issue 2, April 2015, Pages 137-144, ISSN 0268-4012, http://dx.doi.org/10.1016/j.ijinfomgt.2014.10.007.

Press, G. (2013) A Very Short History of Big Data. Retrieved from http://www.forbes.com/sites/gilpress/2013/05/09/a-very-short-history-of-big-data/#1a7787af55da

Walker, M. (2014). Best Practices for Successfully Leveraging Enterprise Architecture in Big Data Initiatives (G00267056). Gartner.

Wikibon. (2012). Taming Big Data. Wikibon. Retrieved September 20, 2017 from https://wikibon.com/taming-big-data-a-big-data-infographic/

 

, ,

1 Comment

Come on… Big Data! Big Data!

The Enterprise Data Architecture, Part 1

 

With the explosive growth of the Internet and more recently, the Internet of Things (the rise of the machines…!!!), there has been a corresponding explosive growth in the amount of data available and collected from all those connected devices and software.  However, along with the vast volume of data that has been generated, there has been an even greater challenge in dealing with the data.  As early as 2001, various IT analysts started reporting on potential issues that would arise due to the mass amounts of data being generated.  Most notably, this article by Gartner analyst, Doug Laney, summarized the various issues attached to Big Data (as it’s commonly called) into whats known as the 3Vs:

  • Data Volume – The depth/breadth of data available
  • Data Velocity – The speed at which new data is created
  • Data Variety – The increasing variety of formats of the data

Now, 16 years after that article was written, we still face many of the same challenges.  Even though technology has improved to be able to handle larger amounts of data processing and analytics, the amount of source data has continued growing at exponential rates.  Cisco releases each year a Visual Networking Index which shows previous years data as well as predictions regarding the next few years as well.  In their 2016 report (Cisco, 2016), the prediction was that by 2020, there would be over 8 billion handheld/personal mobile ready devices as well as over 3 billion M2M devices (Other connected devices such as GPS, Asset Tracking Systems, Device Sensors, Medical applications, etc.) in use, consuming a combined 30 million terabytes of mobile data traffic per month.  And that is just mobile data.

   

So what are the real challenges being faced due to this exponential growth of data?  Here are some facts to consider as posted by Waterford Technologies:

  • According to estimates, the volume of business data worldwide, across all companies, doubles every 1.2 years.
  • Poor data can cost businesses 20%–35% of their operating revenue.
  • Bad data or poor data quality costs US businesses $600 billion annually.
  • According to execs, the influx of data is putting a strain on IT infrastructure. 55 percent of respondents reporting a slowdown of IT systems and 47 percent citing data security problems, according to a global survey from Avanade.
  • In that same survey, by a small but noticeable margin, executives at small companies (fewer than 1,000 employees) are nearly 10 percent more likely to view data as a strategic differentiator than their counterparts at large enterprises.
  • Three-quarters of decision-makers (76 per cent) surveyed anticipate significant impacts in the domain of storage systems as a result of the “Big Data” phenomenon.
  • A quarter of decision-makers surveyed predict that data volumes in their companies will rise by more than 60 per cent by the end of 2014, with the average of all respondents anticipating a growth of no less than 42 per cent.
  • 40% projected growth in global data generated per year vs. 5% growth in global IT spending.

Datamation put together this list if Big Data Challenges of which I want to highlight a few a specifically.

  1. Dealing with Data Growth – As already mentioned above, the amount of data is growing year over year.  So a solution that may work today, may not work well tomorrow.  Investigating and investing in the technologies that can grow together with the data is critical.
  2. Generating Insights in a Timely Manner – Generating and collecting mass amounts of data is a specific challenge that needs to be faced.  But more importantly, what do we do with all that data?  If the data being generated is not being analysed and used to benefit the organization, then the effort is being wasted.  New tools to analyze data are being created and released annually and these need to be evaluated to see if there are organizational benefits to be gained.
  3. Validating Data – Again, if your concern is generating and collecting mass amounts of data, then just as important as processing and analyzing the data, it is important to verify the integrity of the data.  This is especially important in the quickly expanding field of medical records and health data.
  4. Securing Big Data – Additionally, the security of the data is a rapidly growing concern.  As seen in recent data hacks such as the Equifax hacking attacks, the sophistication of hacking and phishing is growing at a rate equivalent to the volume of big data itself.  If the sources of data are not able to provide adequate security measures, the integrity of the data can be questioned as well as many other issues.

Big Data is here to stay, and it’s only going to get bigger.  Is your company ready for it?

 

References:

Cisco (2016). Cisco Visual Networking Index. Retrieved from http://www.cisco.com/c/en/us/solutions/service-provider/visual-networking-index-vni/index.html

 

, ,

No Comments

The Future of the Cloud

The Enterprise Application Architecture, Part 3

So technology has been moving to the “cloud” over the past number of years.  More and more of the applications, hardware and devices we use all utilize distributed technology to some degree or another.  Each year, top technology companies predict trends for the growth of cloud technologies.  It was interesting to see what some of the 2017 predictions have been so far.


IDC (International Data Corporation):

Total spending on IT infrastructure products (server, enterprise storage, and Ethernet switches) for deployment in cloud environments will increase 15.3% year over year in 2017 to $41.7B. IDC predicts that public cloud data centers will account for the majority of this spending ( 60.5%) while off-premises private cloud environments will represent 14.9% of spending. On-premises private clouds will account for 62.3% of spending on private cloud IT infrastructure and will grow 13.1% year over year in 2017.

Source: Spending on IT Infrastructure for Public Cloud Deployments Will Return to Double-Digit Growth in 2017, According to IDC.


Gartner:

Gartner predicts the worldwide public cloud services market will grow 18% in 2017 to $246.8B, up from $209.2B in 2016. Infrastructure-as-a-Service (IaaS) is projected to grow 36.8% in 2017 and reach $34.6B. Software-as-a-Service (SaaS) is expected to increase 20.1%, reaching $46.3B in 2017.

Source: Gartner Says Worldwide Public Cloud Services Market to Grow 18 Percent in 2017.


Forrester:

  • Cloud computing has been one of the most exciting and disruptive forces in the tech market in the past decade. And it’s no longer an adjunct technology bolted onto a traditional infrastructure as a place to build a few customer-facing apps. Cloud applications (SaaS), business services, and platforms (IaaS/PaaS) now power a full spectrum of digital capabilities, from the core enterprise systems powering the back office to the mobile apps delivering new customer experiences.
  • The cloud market will accelerate even faster in 2017. Enterprises use multiple clouds today, and they’ll use even more in 2017 as CIOs step up to orchestrate cloud ecosystems that connect employees, customers, partners, vendors, and devices to serve rising customer expectations.
  • Some will push further, shifting from being cloud adopters to becoming cloud companies themselves. Following early examples like GE or Bosch, these companies will become stewards of their own client and product ecosystems.
  • Cloud expansion will exacerbate the cloud management challenge, pushing CIOs to also aggressively tap new and maturing enterprise-grade security, networking, and container solutions

Source: 2017 Predictions: Dynamics That Will Shape The Future In The Age Of The Customer


Oracle:

  • Cloud-based mission-critical workloads will take off.  Cloud has long promised the migration of all enterprise production workloads. But that migration has yet to happen. The chief barrier to cloud migration remains a lack of commitment and recourse to support production service-level agreements. On one hand, cloud providers are limiting their accountability as they lack the talent to support custom portfolios. On the other, they’re failing to provide sufficient control into the public data center to self-manage service-level agreements. The IaaS provider best equipped to take more responsibility and deliver the control tenants demand will be the one to drive cloud migration in 2017.
  • Corporate-owned data center numbers will plummet.  As organizations focus their IT spending on cloud computing, they’ll begin to migrate their workloads from corporate-owned data centers to purpose-built facilities, managed and run by enterprise cloud providers. Mark Hurd predicts that we’ll see corporate-owned data center numbers fall 80 percent by 2025, and that the same percentage of IT spending will be devoted to cloud services.
  • Enterprise cloud becomes the most secure place for IT processing.  This year’s threat landscape will be highly changeable. External threats—coupled with the need for better governance and privacy mandates—will make security a key priority for all lines of business. In years past, security was a major barrier to cloud investment. Data sovereignty, data privacy, and control issues deterred many organizations from pursuing cloud adoption. But in the future, those very same concerns will be the things that draw new organizations to the cloud.
  • Digital Transformation becomes the norm.  Our world is becoming increasingly digitally connected, and it’s transforming the way we live, work, and play. These same technological advancements provide unprecedented opportunities for businesses to expand, innovate, and create new value. Sectors including healthcare, manufacturing, and even urban planning have been reimagined and redefined by the cloud. To realize these opportunities, today’s enterprises must not only develop new cloud-ready tools, but also put digital at the center of their businesses. Hidden within today’s digital connections are the solutions to our most urgent business challenges.
  • The Rise of Intelligent Applications.  Artificial intelligence (AI) might sound like science fiction, but many of us use it every day. The software behind many online shopping sites and on-demand music services, for example, is a highly successful and highly pervasive form of AI. These systems depend on technology infrastructure capable of importing, analyzing, and interpreting huge volumes of data before acting on it—all without human intervention. And the next step for such technologies? To become an established part of customer service and other business operations.
  • AI gets real.  AI and robotics have carved out a niche in the manufacturing sector, and now these technologies are poised to bring their exciting benefits to a host of new industries. The AI space is white-hot, and it’s being fueled by the data explosion. Machine learning algorithms find patterns in enormous volumes of digital information and use that data to train, learn, and become even smarter. CIOs ignore the AI wave at their peril. According to Toby Redshaw, consultant and former American Express CIO, the company that ignores AI-powered technology will be “the guy at the gunfight with a knife.”
  • Developers do more with less coding.  This year, a new tool looks set to join cloud app developers’ toolkits. “Visual” or “low” coding will be everywhere in 2017. For many organizations, the real-time enterprise has meant a rethink of application development. IT teams are often stuck with a backlog of work, preventing them from delivering applications quickly enough to capitalize on new opportunities. Visual coding enables quick, straightforward development and extension of enterprise applications. “More than ever before, application development and delivery professionals must obsess over their UI designs,” say Forrester analysts John Rymer and Clay Richardson. “Low-code vendors employ familiar drag-and-drop, WYSIWYG techniques to speed user interface creation.”
  • The Cloud empowers small business innovation.  Cloud has become a catalyst for small business growth, allowing them to innovate freely, carve out new markets, and disrupt the status quo. The digital economy demands that companies of all sizes compete based on technology-enabled value. While some seek to evolve existing business practices, others are striving to launch new services that exploit extensive, low-cost computational power. Traditionally, access to such high-performance resources has been too expensive for smaller businesses. But what once cost 100 million USD up front is now available for 10 USD per hour.
  • 60 percent of IT organizations move systems management to the cloud.  More than 90 percent of companies have multiple systems management tools, but just six percent trust their incomplete data. Consequently, IT operations professionals struggle to create effective management approaches. The pace of business is increasing. As more organizations adopt DevOps practices and focus on digital experience, they’ll need to eliminate management data silos and embrace machine learning just to keep up. Some have already embraced systems management in the cloud, unifying management data across multiple clouds and on premises. Others are benefiting from data science applied to the operational management problem. Only Oracle Management Cloud provides an intelligent, unified, cloud-based approach that applies machine learning to the complete operational data set. And while many cloud tools are built exclusively for cloud systems, ours does both.
  • 50 percent of DevTest will move to the cloud.  Last year, we predicted that DevTest workloads would have all but completely migrated to the public cloud by 2025. At Oracle OpenWorld 2016, Mark Hurd revealed that “we are nearly halfway there” already. With on-premises hardware and software, IT teams have to buy, license, and configure everything to create development environments that hopefully match production environments. Hurd estimates that the industry could save 150 billion USD by migrating DevTest to the cloud.

Source: Oracle Cloud Predictions 2017


 

, ,

No Comments

Service Oriented Anarchy

The Enterprise Application Architecture, Part 2

SOA.  Sons of A….   I mean Service-Oriented Architecture.  What exactly is this?  And what’s the big deal?

In Part 1, I talked about Software as a Service, and mentioned Platform as a Service as well as Infrastructure as a Service.  Each of those strategies is taking a commonly used aspect of technology and converting it to a service that can be accessed over the Internet.  SaaS, as I mentioned in Part 1, is about delivering an application over the Internet to an end consumer, or simply, a distribution method. SOA, on the other hand, is architectural approach for application development, or a development method.  By converting all parts of a system to act as independent services, then those same pieces can be used by multiple applications, all having a common framework for communication, referred to as a service bus.  These pieces become the building blocks for a larger system, enabling the parts to easily interact with one another.

IBM uses a combination of multiple definitions to elaborate on SOA :

The more you search around the Internet, the more varied and diverse definitions you find for SOA.  As Martin Fowler, software developer, author and Chief Scientist for ThoughtWorks (a technology consulting company), says in his 2005 article Service Oriented Ambiguity:

  • For some SOA is about exposing software through web services. This crowd further sub-divides into those that expect the various WS-* standards and those that will accept any form of XML over http (and maybe not even XML).
  • For some SOA implies an architecture where applications disappear. Instead you have core services that supply business functionality and data separated by UI aggregators that apply presentations that aggregate together the stuff that core services provide.
  • For some SOA is about allowing systems to communicate over some form of standard structure (usually XML based) with other applications. In it’s worse form this is “CORBA with angle brackets”. In more sophisticated forms this involves coming up with some form of standard backbone for an organization and getting applications to work with this. This backbone may or may not involve http.
  • For some SOA is all about using (mostly) asynchronous messaging to transfer documents between different systems. Essentially this is EAI without all the expensive EAI vendors locking you in.

So do we have a better understanding of SOA?  Or is it chaos and anarchy?  I think his definition by Oracle was simple enough for a begining understanding: “A service-oriented architecture is a way of sharing functions (typically business functions) in a widespread and flexible way.”

 

, ,

No Comments