Home‎ > ‎Architecture‎ > ‎

Performance Modeling Process in IT Systems

posted Sep 29, 2018, 12:47 PM by Jageshwar Tripathi   [ updated Jan 14, 2019, 9:09 AM ]
Performance of software systems is an important factor. Often performance issues in software applications are identified very late (mostly reported by performance testing team or from the production by end users). It is too late to fix the performance issues because by the time it is reported application architecture, design and code is completed. Any changes require huge cost mainly due to impact of changes and testing. It is a good practice to design the systems for performance and to do that it is important to know: 
  • Performance objectives
  • How to design for those objectives and 
  • Before making any investments based on the design approach, understand how the system will perform under expected load (against performance objectives)  
Performance modeling helps understanding how an application will perform under load. Some simulation techniques can be performed to define the model and discover how application will perform under certain conditions. Collection of data under current conditions or data collected from history is required for some of the models. 

Performance modeling process

Identify key scenarios

Identify those scenarios in which performance is important and the ones that pose the most risk to your performance objectives.

Identify workloads

Identify how many users, and how many concurrent users, your system needs to support.

Identify performance objectives 

Define performance objectives for each of your key scenarios. Performance objectives reflect business requirements.

Identify budget

Identify your budget or constraints. This includes the maximum execution time in which an operation must be completed and resource utilization such as CPU, memory, disk I/O, and network I/O constraints.

Identify processing steps

Break your scenarios down into component processing steps.

Allocate budget

Spread your budget determined in Step 4 across your processing steps determined in Step 5 to meet the performance objectives you defined in Step 3.

Evaluate

Evaluate your design against objectives and budget. You may need to modify design or spread your response time and resource utilization budget differently to meet your performance objectives.

Validate

Validate your model and estimates. This is an ongoing activity and includes prototyping, testing, and measuring.

Types of performance modeling

It is important to note that every model is not suitable for every application and its context so choosing right model is the key. It is common mistake to select wrong model and realizing later that modeling didn't work but actually it is wrong model which didn't work. 

Integrated performance model

Integrating performance modeling with development process is one of the way and probably one of the precise way  but this may not be possible in all cases (e.g when you don't have control on development of software, packaged solution etc). It also requires effort and time and is a significant investment. This is why many application development teams (owners of those applications) avoid this exercise. There are tools provided by development tool (IDE) vendor and there are tools developed by third parties. 

Applicability:

When performance is important focus for business/users
When application development is under your control

Pros:

Precise

Cons:

Requires significant amount of effort and time. 

Component based performance model

In cases where tightly integrated performance modeling is not the choice, there are other ways to conduct performance modeling. 

Applicability:

Packaged software
When application is divided in to independent components 
When cost/effort is limited

Pros:

Simple and easy to apply
Less effort consuming

Cons:

May not provide very precise inputs to the developers or architects

Approaches:

Discrete event

Used in integrated or tightly coupled modeling.

Analytical

These are simulation based models where a simulator simulates behavior of application component in consideration.

Statistical

These models use statistical techniques. Observation results are plotted for different conditions of usage and load. 

Interesting resources

1. http://jmt.sourceforge.net/
2. https://www.palladio-simulator.com/tools/
3. http://www.perfdynamics.com/Tools/PDQ.html
4. https://www.dynatrace.com/news/blog/how-to-create-performance-models-using-application-monitoring-data/
5. https://www.vtt.fi/inf/pdf/publications/2003/P512.pdf
6. https://waset.org/publications/3902/performance-modeling-for-web-based-j2ee-and-.net-applications
7. https://pdfs.semanticscholar.org/d688/37cbfb63c5c3469a7db7c520881001a039b8.pdf
8. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.10.2690&rep=rep1&type=pdf
9. http://alicezheng.org/papers/sigmetrics10-practicalperf.pdf
10. http://www.archer.ac.uk/training/course-material/2018/07/ScaleMPI-MK/Slides/PerformanceModelling.pdf
11. https://hal.archives-ouvertes.fr/hal-01118352/document
12. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.187.3046&rep=rep1&type=pdf
13. http://www.scs-europe.net/dlib/2013/ecms13papers/hipmos_ECMS2013_0178.pdf
14. http://www.iaeng.org/publication/WCECS2010/WCECS2010_pp35-43.pdf
15. http://jmt.sourceforge.net/Papers/acm09jmt.pdf

Comments