Performance Modeling Process in IT Systems
Post date: Sep 29, 2018 7:47:40 PM
Performance of software systems is an important factor. Often performance issues in software applications are identified very late (mostly reported by performance testing team or from the production by end users). It is too late to fix the performance issues because by the time it is reported application architecture, design and code is completed. Any changes require huge cost mainly due to impact of changes and testing. It is a good practice to design the systems for performance and to do that it is important to know:
Performance objectives
How to design for those objectives and
Before making any investments based on the design approach, understand how the system will perform under expected load (against performance objectives)
Performance modeling helps understanding how an application will perform under load. Some simulation techniques can be performed to define the model and discover how application will perform under certain conditions. Collection of data under current conditions or data collected from history is required for some of the models.
Performance modeling process
Identify key scenarios
Identify those scenarios in which performance is important and the ones that pose the most risk to your performance objectives.
Identify workloads
Identify how many users, and how many concurrent users, your system needs to support.
Identify performance objectives
Define performance objectives for each of your key scenarios. Performance objectives reflect business requirements.
Identify budget
Identify your budget or constraints. This includes the maximum execution time in which an operation must be completed and resource utilization such as CPU, memory, disk I/O, and network I/O constraints.
Identify processing steps
Break your scenarios down into component processing steps.
Allocate budget
Spread your budget determined in Step 4 across your processing steps determined in Step 5 to meet the performance objectives you defined in Step 3.
Evaluate
Evaluate your design against objectives and budget. You may need to modify design or spread your response time and resource utilization budget differently to meet your performance objectives.
Validate
Validate your model and estimates. This is an ongoing activity and includes prototyping, testing, and measuring.
Types of performance modeling
It is important to note that every model is not suitable for every application and its context so choosing right model is the key. It is common mistake to select wrong model and realizing later that modeling didn't work but actually it is wrong model which didn't work.
Integrated performance model
Integrating performance modeling with development process is one of the way and probably one of the precise way but this may not be possible in all cases (e.g when you don't have control on development of software, packaged solution etc). It also requires effort and time and is a significant investment. This is why many application development teams (owners of those applications) avoid this exercise. There are tools provided by development tool (IDE) vendor and there are tools developed by third parties.
Applicability:
When performance is important focus for business/users
When application development is under your control
Pros:
Precise
Cons:
Requires significant amount of effort and time.
Component based performance model
In cases where tightly integrated performance modeling is not the choice, there are other ways to conduct performance modeling.
Applicability:
Packaged software
When application is divided in to independent components
When cost/effort is limited
Pros:
Simple and easy to apply
Less effort consuming
Cons:
May not provide very precise inputs to the developers or architects
Approaches:
Discrete event
Used in integrated or tightly coupled modeling.
Analytical
These are simulation based models where a simulator simulates behavior of application component in consideration.
Statistical
These models use statistical techniques. Observation results are plotted for different conditions of usage and load.
Interesting resources
1. http://jmt.sourceforge.net/
2. https://www.palladio-simulator.com/tools/
3. http://www.perfdynamics.com/Tools/PDQ.html
4. https://www.dynatrace.com/news/blog/how-to-create-performance-models-using-application-monitoring-data/
5. https://www.vtt.fi/inf/pdf/publications/2003/P512.pdf
6. https://waset.org/publications/3902/performance-modeling-for-web-based-j2ee-and-.net-applications
7. https://pdfs.semanticscholar.org/d688/37cbfb63c5c3469a7db7c520881001a039b8.pdf
8. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.10.2690&rep=rep1&type=pdf
9. http://alicezheng.org/papers/sigmetrics10-practicalperf.pdf
10. http://www.archer.ac.uk/training/course-material/2018/07/ScaleMPI-MK/Slides/PerformanceModelling.pdf
11. https://hal.archives-ouvertes.fr/hal-01118352/document
12. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.187.3046&rep=rep1&type=pdf
13. http://www.scs-europe.net/dlib/2013/ecms13papers/hipmos_ECMS2013_0178.pdf
14. http://www.iaeng.org/publication/WCECS2010/WCECS2010_pp35-43.pdf
15. http://jmt.sourceforge.net/Papers/acm09jmt.pdf