Business Intelligence Tools and Techniques

4_Data Mining Predictive Analysis of Stocks_6MinuteDream

2008-09-29T09:50:00.000-07:00

Project Name: Predictive Modelling of Stock Markets

Team Number: 4 (G Raviteja, Manmohan Agarwal, Ankit Gupta, D Sushanth Reddy)

Team Name: El Matador

Problem Statement: What is the effect of the change in the price of a particular stock over other stocks present in a particular Index or over various indexes or for that matter over different markets, over different time periods?

Data Source: BSE SENSEX and NIFTY websites provide an ample resource of daily stock related data for any individual stock. Also the data could be obtained from the financial portals like moneycontrol.com

The Benefit / Utility: The data analysis would reveal patterns of correlation between individual stocks. The correlation could be positive or negative or even neutral. The correlation patterns obtained could be a base for different types of investors in making their financial investments. The target set of investors could be an individual investor, an institutional investor, an investment bank, a hedge fund firm, a PE player, and market regulators like SEBI etc.

Individual Investor – Individual investors, by nature of being risk averse, would like to invest or park his money in safer stocks. If he/she is confident that a particular stock is safe to invest in, the analysis would provide him with a option of a diversified portfolio wherein he can hedge his risk.

Institutional Investor – Institutional investors create a portfolio of different set of stocks i.e they try to invest their money into stocks from different sectors, stocks of companies of different sizes etc. Our analysis could provide a stronger foundation to the investment decisions of the institutional investors and also allow them to plan their investments in a way that they could minimise their risk.

Hedge Funds – Hedge funds usually offset their potential losses by hedging their investments, notably by short selling. Since we are analysing stock correlations across different time periods, we feel that the short term data analysis would be helpful for such companies.

Market Regulators – The market regulators could keep a track on the movement of shares and an uneven spike or a correlation that is not normal could be investigated.

Expected Outcomes: The relationships between stocks are either positively, negatively or neutrally correlated to each other as an exhaustive set. The outcome gives as an overview of how different stocks change in their relation to other stocks in different length of time, across different parameters, different markets. It provides you with an overall bird’s eye view as well as a very specific view as what happens to the change in a particular day for a particular stock. The outcome would be an exhaustive set which caters to any kind of requirements for any kind of users who play in the equity markets. This model may be further extended to even derivative markets which are now a days termed as the ‘Weapons of Mass Destructions’ and can be used as a effective defensive tool against them.

2_Data Mining in Sports_Wanderers

2008-09-29T09:36:00.000-07:00

Project Name: Data Mining in Sports
Team Number 2
Team Name: Wanderers
Team Members:
1. Bhuvan Prasad A
2. Akshat A Patil
3. Guruprasad A Shenoy
4. Ankit Anand

The Problem Statement: Does past performance predict future performance? Using statistical data about the past, is it possible to build a model of predictive value?

Data Source: Cricket archives on websites (cricinfo.com) and related sites
The Benefit / Utility: Considering that IPL has finally arrived with a bang, we need to consciously think about sports as a commercial enterprise. This also means, we need to give prediction a thought. It can be used to predict the future gains from a particular player while you buy the player in an auction. Prediction can also be used in betting on websites like Ladbrokes.com

Expected Outcomes: We hope to gain insight into predictive analysis by finding patterns in games using quantization of player and team attributes. We plan to use previous match records to improve upon our model by including hidden factors on a continuous basis and comparing it with current records.

1_housing_6MinDream

2008-09-29T05:58:00.000-07:00

Project Name: Housing
Team number 1
413/15 Arjun A V
401/15 Abhay Kesharwani
416/15 Atul Kumar
426/15 Joy Bhagat

Problem Statement: Housing rates in suburban Houston

Data Source:
Origin:

This dataset was taken from the StatLib library which is maintained at Carnegie Mellon University.

Creator:

Harrison, D. and Rubinfeld, D.L.
'Hedonic prices and the demand for clean air', J. Environ. Economics & Management, vol.5, 81-102, 1978.

Benefits/utility : The major benefit would be to determine the various factors and study their impact on prices of house. This can give a good example of what people look for wen buying/renting a house. Also the fact that the data set has hedonistic measures apart from social measures allows us to look the problem from a behavioral point of view. Also the given dataset and the parameters can be used as a base model for other geographic locations.

Expected Outcomes :We believe using the given data set on stastica we can make a model to determine the house prices and by this we can study the real estate market depending on market prices. Also similar data sets for different locations can give us location specific models. Although the given data-set is old but considering the current real estate market post Sub prime the model can give fair valuation for the housing prices.

9_BankRegulationandSupervision_6mindream

2008-09-29T02:34:00.000-07:00

Bank Regulation and Supervision

Project Name: Bank Regulation and Supervision

Team Members of Group 9 (Abhinav Mathur, Mohit ,Lokesh, Sauravl)

Problem Statement: To analysis how various regulation affect the performance of bank and factors that lead to the formulation of such regulation in various countries.

Data Source:

1. http://econ.worldbank.org/WBSITE/EXTERNAL/EXTDEC/EXTRESEARCH/0,,contentMDK:20345037~pagePK:64214825~piPK:6421494

2. http://dbie.rbi.org.in/

Benefits and Utility: The basic aim of this project is to find out how various policies taken by the government affect the functioning of banks. As a result of this banks need to change various ratios and rates. This in-turn affects various social development program and other business activities and Entrepreneurship activities. This work will also help bank analyze and prepare various contingency plan when policies are framed by the government to promote various scheme to promote particular sector. This will subsequently increase the profitability of the bank. As a result these banks can be channelized in funding various government and other projects and expand the scope of their operation. We also need to know various factors that government consider in order to formulate the regulatory policies.

Expected Outcome:

· Comparative study about various decision in different countries.

· Which policies are suitable for which type of economy and the state of development

· How various investors and entrepreneurship decide about using these policies to expand their domain of business.

7_Impact of Achievements in sports on Macro Economic Indicators of a country_6 Min Dream

2008-09-29T01:14:00.001-07:00

Impact of Achievements in sports on Macro Economic Indicators of a country

Group 7

· Jose Vinay (425/15)

· Prashant Pande (438/15)

· Poonacha KM (440/15)

· Rittu C Joseph (446/15)

Problem Statement

· Is it a pure coincidence that a few countries have seen a positive impact of achievements in sports on their economy?

· If yes, is there some pattern or some correlation which exists?

· If there is any impact, does a team achievement have a greater impact or an individual achievement?

· How big an impact can the economy see if it bags contracts of hosting major events like Olympics, Common Wealth Games or the Asian Games?

· How much impact does the magnitude of the sport event have on the country?

· Who are more sensitive to such events - developed, developing or the underdeveloped countries?

· Also looking forward to explore a whole lot of relationships which could exist between achievements in sports and the overall economy.

Data Source

The trends and variations of the various Macroeconomic factors of different countries are available on the websites of Asian Development Bank, Wikipedia and other sites over the internet.

The achievements of different countries in sports are available on a host of news feeds (BBC, CNN, NDTV etc) and sports web sites (ESPN Start Sports, Cricinfo etc).

Benefits

Government: We expect that the outcome of this report would help government bodies to decide upon the attention that sports should get. A strong dependency will encourage the government to pay more heed to sports.

Sports Authorities: Sports authorities can use the outcome of this report for raising funds.

Investors: It will also be an important piece of information for investors and the common man, this can act as a straw in the wind to help them identify appropriate times to enter the market.

Basically, a positive outcome of the report could help in increasing the awareness among people about the importance of sports.

Expected Outcomes

At this point of time, the relation between these two factors appears to be loose, if at all it exists. However there have been sporadic instances which hint towards the existence of a positive impact which achievements in sports have on the Macroeconomic factors. The project aims at unearth this relation if it exists.

6_DM of IIMC Grades_6 Min Dream

2008-09-28T21:52:00.000-07:00

Project Name: DM in IIM Grades

Team Number: 6

Team Members:

461/15 – Hrishikesh Thite

460/15 – Tarun Gupta

457/15 – Sreekanth Reddy

462/15 – Tushar

Problem Statement:

Data Mining and Analysis of grades of the students at IIM Calcutta across terms, courses, years and correlation with their previous and possibly future performance, with the evaluation process

Data Source:

The PGP Office will supply the grade data, presumably in an Excel format

In detail, we will get data from the PGP Office that has course-wise scores broken up by each of the evaluation criteria. This will be mined for trends across courses, across years, across students, across evaluation methodologies, across faculty, across scores in individual tests versus when they were conducted (in the course timelines) and so on. Hopefully, some interesting ideas will come to light.

Benefit / Utility:

There are many stakeholders in such a study:

· Students: Trends mined will allow students to select their basket of courses (assuming that they want to maximize their scores). Students will also be able to profile themselves based on their past performance and get a prediction on their CG at IIMC.

· Course Coordinators: They will be able to figure out the most effective evaluation methodology across courses (since we will correlate scores/grades with respect to no. of quizzes, exams, projects, reports etc. and the performance of the students in each of them)

· PGP Office: Popularity across courses. Evaluation of faculty on the basis of performance of their students, isolated in a particular faculty’s course versus overall performance of the student across all courses.

· Faculty: Student feedback as measured by their performance in the course versus the overall performance across all courses.

· IIMC Branding: IIMC is often regarded as a Quant heavy campus. Is this really true? Is it because students perform better in quant courses? Or are they predisposed to quant studies?

Outcomes:

From the above exercise, we can publish two-three sets of analysed data – a student-edition, a faculty-edition and a PGP-office edition. Each will contain trends relevant to the target stakeholders.

Following are some of the trends we expect:

· Absolute grades to improve over time – relative grades to remain more or less the same

· Regular quizzes and projects with lesser emphasis on end-terms etc. will show better performance among students with a smaller spread across the scores

· Fin-focused subjects to show up in the popularity pattern

· DCM / MIS courses to show higher grades

· Some sort of trend in the soft versus hardcore courses, not sure which way this will go

· More popularity of the high scoring courses

Entrepreneurship

2008-09-28T21:10:00.000-07:00

Project Name: Entrepreneurship: Factors leading to an entrepreneurial culture

Team Number: 13 (Rohit Kwatra, Anmol Singla, Manan Mehta, Amritanshu Kumar)

Team Name: Team Calvin

Problem Statement: What combination of factors lead to an entrepreneurial culture?

Data Source: 2007 World Bank Group Entrepreneurship Survey measuring entrepreneurial activity in 84 developing and industrial countries over the period 2003-2005

The Benefit / Utility: The dataset analysis will reveal important drivers for entrepreneurship across countries and its related effect on economic development. As a result it can help economists and administrators give a better idea about the relationship between entrepreneurial activity and other indicators such as economic and financial development and growth, the quality of the legal and regulatory environment etc. The entry rates of new firms (defined as newly registered firms as a percentage of total registered firms in the previous year) range between 7 and 9 percent among various regions. Also provided are data pertaining to electronic business registration, which is shown to be related to lower costs and a shorter number of days required to start a business, highlighting the impact of regulatory, political, and tax changes on new firm registrations. Thus the new data provides managerial perspectives on this aspect, raising questions about the effects of changes in the business environment on entrepreneurial risk taking, and about reforms which will spur faster firm registration. It may also help entrepreneurs looking at the results, in relating to and identifying reforms that promote higher growth in the sector.

Expected Outcomes: We expect that the results would provide better understanding about the factors that contribute to greater entrepreneurship, formal sector participation and the impact of related policy reforms. These results can guide effective policymaking and deliver new capabilities for identifying the impact of reforms. Although we find significant relationships with these measures – i.e. more dynamic economies in countries with better business environments – we cannot postulate on the direction of causality. This survey will provide a new set of indicators to study the relationship between business creation, the investment climate, and economic development. Expectations are that a higher level of entrepreneurship significantly relates to greater economic development, formal sector participation, and better governance. For instance, countries with lower barriers to entry and less corruption generally should see higher percentages of firm registrations and entry. This might suggest that countries that facilitate entrepreneurship see commensurate increases in overall economic growth and an expansion of the formal sector. Alternatively, it might be the case that periods of economic expansion encourage optimism and entrepreneurship; for instance, individuals might be willing to leave their job security to start a business if they are more confident they could find another job if their business fails. We hope to analyse these factors, which will allow us to better understand how the private sector behaves over business and financial cycles. Furthermore, entrepreneurship indicators can be used to complement the development of policy recommendations to promote private sector development and growth. Moreover, the process of collecting data will become a valuable tool for the diagnosis of the business environments. For instance, direct contact with business registries in more than a 125 countries will help us to better understand the difficulties that entrepreneurs face when incorporating a business, as well as the impact of the institutional and technological framework of registries in the ease of starting a business. On the whole, we will be able to link the outcomes with our expectations regarding the entry barriers a business faces while entering a new region. Though these findings and our expectations should be line, but any deviations will be better looked upon through this data and the reasons behind them analysed.

The Role of Macroeconomic Factors in Growth

2008-09-28T20:57:00.000-07:00

Group 10
Abhinay Puvvala (FP/12)
Aman Goel (403/15)
Ayush Garg (417/15)
Harshit Duggal (424/15)

Problem Statement

The purpose of this project is to find out the correlation between various economic factors and growth of a country. We would be looking at basic Macroeconomic factors like inflation, large budget deficits, and distorted foreign exchange markets and how these effect the growth of a small and a large country.

Dataset and our Approach

We will work on a comprehensive data set of 122 Countries and analyze them on the basis of 15 variables spread over 28 years. The data set contains the various macroeconomic variables like inflation, large budget deficits and foreign exchange for these countries. We will analyze the data with various statistical techniques such as regression, correlation and aim to find out causal relationship between macroeconomic factors and growth.

Benefits

This study will help economists in framing the economic policies. They can understand the effect of any decision taken on long term health of a country. They can see the cross affect of the economic factors and design policies for long term growth of the economy. The size of the country also plays an important role in analyzing these factors.

Expected Outcomes
Inflation: At least theoretically Higher Inflation rates have a negative correlation towards growth rate. The basic goals for including this Macroeconomic factor in our project for analyzing the growth rate are
Is there an empirical relationship between growth and inflation?
Is the relationship stable across countries and across time periods?
Is the relationship structural?
Does the empirical relationship show that there is an exploitable trade-off by monetary policymakers?
If there is an exploitable trade-off, what are the welfare implications of that trade-off and what is the optimal rate of inflation?

Budget Deficits: Conventional analyses of sustained budget deficits demonstrate the negative effects of deficits on long-term economic growth. Under the conventional view, ongoing budget deficits decrease national saving, which reduces domestic investment and increases borrowing from abroad. The reduction in domestic investment and the increase in the current account deficit both reduce future national income, with the loss in income steadily growing over time. Under the conventional view, the costs imposed by sustained deficits tend to build gradually over time. Substantial ongoing deficits can generate a self-reinforcing negative cycle among the underlying fiscal deficit, financial markets, and the real economy.

Foreign exchange market: We felt this as a very important factor because exchange rate of a currency generally reflects huge and diverse economic factors (Surplus/deficit budgets, Balance of trade, Inflation levels, Economic Health) apart from Political Conditions and Market Psychology. The above mentioned factors also directly or indirectly have a bearing on the growth of an economy. We would try to bring out how distorted foreign exchange markets can be detrimental for the growth of an economy.

11_An Integrated Administration Process @ IIMC_6 Min Dream

2008-09-28T20:22:00.000-07:00

Project Name: An Integrated Administration Process @ IIMC

Team no: 11

Members: Anirvan Sarkar (406/15),

Ashutosh Gadodia (415/15),

Sayuz Basak (453/15),

Sourav Saha (FP/06/2008)

Problem Statement: The different academic programs in IIMC works independently of each other with minimal and sometimes even no-interaction among them. The aim is to try address this loophole in the IIMC student administration process and suggest a solution (preferably system based) in order to counter the same.

The 3 factors are:

1. Create dependence among the PGP, PGDCM, FP & PGPEX system

2. Design a system to help data flow among the different program office

3. Foster student interaction and ideas generation across the programs

Data Source: The PGP Office, the FP office, the PGPEX office in IIMC. The student council members across the different programs

Benefits: Both the students as well as the administration would be beneficial from the above proposal. We aim to optimize valuable resources like faculty-lectures, class-rooms, staffs and most importantly students’ experience and time spent in IIMC.

Expected Outcomes: Though the problem looks pretty straightforward but the solution is not. Expected outcome would be balanced class-loads, more socialization events among students and substantial free-slots to shape ones own hobbies and dreams.

3_Facebook_6 Min Dream

2008-09-28T17:06:00.000-07:00

1)Project Name: Facebook

Team Number:3

Team Name: Data Networking

Team Members: Ankit Singhal, Anand Justin Cherian, Ravi Gupta & Sushim

2)The Problem Statement: To find out interaction patterns in Online Social Networks

3)Data Source: Dummy Data from: http://developers.facebook.com/fbopen

4)The Benefit/Utility: There are multiple uses of the findings we aim to gather from our project. The primary and straightforward use is by the online advertising companies. By establishing a conclusive link between online users and their behavior, companies can use innovative specific targeting Ads online. Other uses include those by the social networking sites themselves to analyze the data and increase their user base through customized marketing and individualized website experience.

5)Expected Outcomes: Expected outcome are that the most active age group on social networking sites like Facebook will be the 18-25 years group. A pattern can also be established in specific countries regarding the usage of the website at specific time patterns. The average time spent by any user on the site is expected to be directly proportional to his/her ‘networking nodes’ and vice versa. To be a bit too specific, single girls are more likely to be interested in advertisements about jewelry and make up than married female users.

12_Does Governance Matter_6 Min Dream

2008-09-28T16:50:00.000-07:00

Project Name: Does Governance Matter?
Team no: 12
Members: Kinjal Sengupta (430/15), Shobhit Bhatnagar (455/15), Nishank Gosain (437/15), Prashant Agarwal (441/15)

Problem Statement: Try and find the relationship between governance and 6 factors generally related to it, and find out what is the correlation between them. The 6 factors are:
1. Voice and Accountability
2. Political Stability and Absence of Violence
3. Government Effectiveness
4. Regulatory Quality
5. Rule of Law
6. Control of Corruption

Data Source: The Worldwide Governance Indicators (WGI) project report: Aggregate and individual governance indicators for 212 countries and territories over the period 1996–2007

Benefits: The growing recognition of the link between good governance and successful development has stimulated demand for monitoring the quality of governance across countries. The project can help us understand what makes government work, what the key factors for its effectiveness are and which factors are not as important as they are generally deemed to be. These indicators can also be used by countries to benchmark themselves against others.

Expected Outcomes: Though the dependence looks pretty direct, we wish to find the exact correlation between these factors and good governance. Secondly we also wish to see, are there any factors which do not affect governance or affect it in a way opposite to what is perceived. Through this we also look forward to identify any outliers i.e. any countries doing poorly on the indices but still having good governance or vice versa

14_Economic Growth and Environmental Quality_6MinDream

2008-09-28T15:25:00.000-07:00

1) Project Name: Economic Growth and Environmental Quality

Team Name: Joka Juggernauts

Team Number: 14

Team Members: Saurabh Sunil, Satwik Sharma, Shweta Poddar, Devvrat Tripathi

2) The Problem Statement: We are trying to find out the linkages, if any, between the growth of the economy and the quality of its environment.

3) Data Source: World Bank Research data of Shafik and Bandopadhyay

4) The Benefit / Utility: As the economy of the developing countries grows, it is putting more and more pressure on the already stretched natural resources. We are polluting more than the earth can possibly consume. What we are trying to find here is the relations that might exist between environmental conditions and growth. We have the data of the macro-economic variables of a lot of countries and the cities in these countries. We also have the data on the pollution levels of these countries. What we are trying to do is to find the possible associations between both the data.

5) Expected Outcomes: The findings may vary. However, this will once and for all answer the million dollar question—“Should we worry about the economic consequences of pollution?”. This will indeed help in framing the policies of various nations.

5_CourseSelection,CGPA,Career_6MinDream

2008-09-28T14:59:00.000-07:00

1) Project Name: Course Selection, CGPA and Career

Team Name: Miners

Team Number: 5

Team Members: Mahesh Chayel, Preet Pillai, Ravi Dilip Kumar, Ashish Kumar Vijan

2) The Problem Statement: How does Course Selection affect CGPA of a student. Does CGPA have a bearing on placement, career and success of a person.

3) Data Source: Extract from PGP Office, Alumni Cell.

4) The Benefit / Utility: The current batch and future students of IIM Calcutta will be the main beneficiaries. The students will be able to choose the courses based upon the CGPAs also in mind while selecting a particular course. We are also planning to look into the previous data and see if the selection of a particular course improved or made the CGPA worse. The other benefits include the possibility of finding relation between the career and the path taken by previous IIMC students based upon the CGPA they had. If we can find clusters of in data, we can also show that a particular set of CGPAs have a certain leaning towards a particular sector in placements.

5) Expected Outcomes: The findings can vary a lot but we think that there will be some correlation between high CGPAs and placements, but we are not sure to what level and above what level. There is also a chance that we can show that taking up a particular course improves the CGPA of students.

8_FinancialService_6MinDream

2008-09-28T14:38:00.000-07:00

Project Name: Financial Service
Team Number: 8
Team Name: KRAPS
Ankit Agrawal (408/15)
Karmendra Jain (428/15)
Piyush Mehta (439/15)
Rahul Sethia (443/15)

The Problem Statement: Finding a link between the access to financial services and the various barriers in building an inclusive financial system.

Data Source: Finance for All? Policies and Pitfalls in Expanding Access, World Bank, November 13, 2007 data link

The Benefit/ Utility: The potential relation between the barriers and the access to financial service can help various countries who have a lower percentage access to financial services in building an effective policy. The relation among the higher access to financial services in certain countries can be used as a model to make effective changes in the countries where these barriers are high and thereby help the population in general because a well-functioning financial system and a vigorous private sector are important drivers of growth and poverty reduction
We would like to indentify how an effective financial system contributes to economic development and identifying which policies work best to improve the efficiency, stability, and reach of the financial system in developing countries. This will help the Governments to strengthen the institutions and help in the growth process such as promoting entrepreneurship, innovation and the process of technology adoption.

Expected Outcomes: We expect that there will be a higher level of access of financial services and lesser barriers among the developed countries like France, USA, UK , while it will be significantly lower among underdeveloped countries like Tanzania ,Liberia etc.
We are also expecting that the countries having higher level of access of financial services will have higher penetration of branches and ATM of financial institutions and lower barriers to loan services. There might be several other factors that will affect the access to financial services.

The Six Minute Dream Project

2008-09-18T06:27:00.000-07:00

That is not to say you will be done and dusted with the project in Six Minutes Flat (unless you are in any way related to this fellow)

The objective of the Six Minute Dream is to get your vision, your idea for the prospective project out in class, to get feedback from your peers and professors.

While you are busy contemplating (gazing pensively into the far distances etc as mentionedbefore) what project to take up, keep in mind, at all time these two points:

Point a)

It should be useful i.e something non-trivial, valuable and even better, counter-intuitive should emerge from your findings. For instance, hours and days of crunching terabytes of financial data on a monster of a data mining package should yield something more substantial than the glorious result "Companies go bankrupt because of lack of money")

Point b)

It should be feasible i.e within the bounds of possibility to complete in one term.

Good quality data must be available (For instance, while "Finding Patterns in Classified Top Secret MOSSAD internal communications about Covert Operations " might seem an interesting project, it won't do well at point b. Unless, of course, you are this guy.)

Every team is expected to come up with a One Page Writeup about their dream project, upload it to this blog and present/discuss the same in class on 1st October. This presentation will be a 6 min communication of your idea to the class and profs. Think of it as a Pre-Proposal for the Project.

No powerpoint slides for class presentation, just coherent ideas.

Guidelines for the One Page Writeup:

Mention:

1) Project Name, Team Number (u can go crazy and call your team a name!) + Team Members

2) The Problem Statement

3) Data Source (preferrably the type of data available)

4) The Benefit / Utility : Who will potentially benefit from the insights mined? How?

5) Expected Outcomes: Gut feeling about what you expect might be the findings.

It is not expected that you know for sure which tool or technique you will use

(of course, if you have some idea, feel free to mention it! - What is important is that you have a fair idea what the problem is that you plan to solve. Know thy pain-point! )

Submission Procedure:

Post this One-Page Write Up on the Blog:

"TeamNo_ProjectName_6MinDream".

Copy-Paste the contents of the OnePage Doc to the Post and Press Publish Post.

(In case of any difficulties with blog access, you can e-mail me the one page doc. Please name the file "TeamNo_ProjectName_6MinDream.doc" )

Timelines:

29 September 09:59:59 - One Page Writeup up on the Blog

1 October - Presentation and Discussion of your idea in Class

Class Lectures and References

2008-09-16T07:05:00.000-07:00

Material referenced during the lectures can all be downloaded from Slideshare:

http://www.slideshare.net/ambujm

Team Co-ordinators for Projects

2008-09-16T05:38:00.001-07:00

This is the list of team co-ordinators for the BITT-1 projects.

Please get in touch with them, form your team (Max 4 members including co-ordinator, per team) and add in your names to this list (edit this post, after logging in to the blog. )

(~ If you haven't yet, email me at myshkinonline (at) gmail (dot) com with your name and reg no to get an invite and editing rights).

Team No.	Team		Project Name
Team No.	Team Co-Ordinator	+ Team Members (max 3 more)	Project Name
1	Arjun A.V – 413/15
2	Bhuvan – 419/15	Akshat Patil (402/15) GuruPrasad Shenoy(423/15) Ankit Anand (409/15)	20) Data Mining in Sports
3	Ravi Gupta – 445/15	Ankit Singhal(411/15) Anand Justin Cherian(405/15) Sushim Gupta(459/15)	21) Facebook Interaction Patterns in Online Social Networks
4	G.Raviteja – 422/15	D Sushanth Reddy - 420/15 Ankit Gupta - 410/15 Manmohan Agarwal - 433/15	Stock Markets - Proj No. 23
5	Mahesh Chayel – 432/15	Preet Pillai - 442/15 Ravi Dilip Kumar - 444/15 Ashish Kumar Vijan - 414/15	TBA
6	Hrishikesh Thite – 461/15	Tarun Gupta - 460/15 Sreekanth Reddy - 457/15	TBA
7	Prashant Pande – 438/15	Jose Vinay C - 425/15 Poonacha K. M.- 440/15 Rittu C. Joseph - 446/15	Correlation between Achievements of a nation with its Macroeconomic factors
8	Karmendra Jain – 428/15	Ankit Agrawal 408/15 Piyush Mehta 439/15 Rahul Sethia 443/15	Financial Services (12)
9	Abhinav Mathur – FP/02/08
10	Abhinay Puvvala – FP/12/08	2. Ayush Garg (417/15) 3. Aman Goel (403/15) 4. Harshit Duggal (424/15)	The Role of Macroeconomic Factors in Growth
11	Saurav Saha – FP/06/08
12	Shobhit Bhatnagar – 455/15	Kinjal Sengupta 430/15 Prashant Agarwal 441/15 Nishank Gosain 437/15	TBA
13	Rohit Kwatra – 448/15	Anmol Singla 412/15 Amritanshu Kumar 404/15 Manan Mehta 434/15	11.) Entrepreneurship
14	Saurabh Sunil – 452/15	Satwik Sharma (450/15 ) Devvrat Tripathi (421/15) Shweta Poddar ( 456/15)

After your team has been formed, think of the project you would like to take on.

Feel free to add it in to "Project Name" as you go along.

Your team will then come and present this (the "dream project"), informally, in class (Prof will confirm a date in class - most likely Monday).

~Happy Digging!

Note: I've learnt there have been some problems editing this post. The problem is I need to grant authorship rights (to create posts) as well as admin rights (to edit other people's posts).

In case you have been sent an invite and still cant edit this post, then accept the invite, log in and hang on - I'll be granting admin rights shortly.

~InefficientWikisAndozz

Getting Started

2008-09-09T00:34:00.000-07:00

There are some truly wonderful resources available on the www. "Why do people share knowledge? Why do experts in the field spend their time and effort making great tutorials and primers, to be given away for free?" is a real question. However, while wiser people are busy solving the mysteries of human motivation, you can get smart on data mining at places like these:

http://dml.cs.byu.edu/wiki/index.php/Data_Mining_Resources
http://it.toolbox.com/blogs/opensource-analytics/database-vs-data-warehouse-8286
http://www.statsoft.com/textbook/stdatmin.html
http://www.kaushik.net/avinash/2007/09/data-mining-and-predictive-analytics-on-web-data-works-nyet.html

Here's a comprehensive list of DM blogs ("Comprehensive listings" and aggregators are in the race to be the "most meta" among all metas, aggregating, and aggregating agregators and so on and so forth...BUT, never mind all that :)...):

http://dataminingresearch.blogspot.com/2008/05/data-mining-blogs-big-list.html

Data Mining is an Adventure. There is a tremendous pleasure in having discovered something that is not apparent, or even better, in having corrected and established as bunk something that seemed initially apparent and so-called "Common-Sense".
For some inspiration, I would recommend you read (and most of you probably have already!) S.Levitt's "Freakonomics"
http://en.wikipedia.org/wiki/Steven_Levitt
http://pricetheory.uchicago.edu/levitt/home.html

Suggested Projects

2008-09-08T00:01:00.000-07:00

The list below is only indicative.
Students are free to choose any other project, or make modifications to the topics/use alternative data sources from those given below
In fact, students are recommended to do their own digging before finalizing on a project. Make sure your team is convinced that something counter-intuitive (!!), non-trivial and useful can be unearthed.

1 ) A Joke of a Project

What can you find by Collaborative Filtering of User Ratings for Jokes? Available: 4.1 Million continuous ratings (-10.00 to +10.00) of 100 jokes from 73,421 users: collected between April 1999 - May 2003.

Dataset: http://goldberg.berkeley.edu/jester-data/

2 ) Open Source Software Development

Study of the Growth Dynamics of OSS

Study of the Social Networks behind OSS Development

Clustering of OSS projects - What are the Salient types of differences, especially with respect to Proprietary Software

Development? Have a look at related work here:http://www.nd.edu/~oss/Papers/papers.html

Dataset available on demand:http://www.nd.edu/~oss/Data/data.html

3 ) Does Governance Matter?

Governance consists of the traditions and institutions by which authority in a country is exercised. This includes the process by which governments are selected, monitored and replaced; the capacity of the government to effectively formulate and implement sound policies; and the respect of citizens and the state for the institutions that govern economic and social interactions among them. Can you uncover insights on the aspects of governance essential for economic growth, for human development, etc?

Dataset: http://info.worldbank.org/governance/wgi/index.asp

4 ) Telecom Regulation

Can you uncover some fundamental insights relating regulatory governance of telcos to performance related parameters of the industry?

Dataset: http://econ.worldbank.org/WBSITE/EXTERNAL/EXTDEC/EXTRESEARCH/0,,contentMDK:20699152~pagePK:64214825~piPK:64214943~theSitePK:469382,00.html

5 ) Patterns in Wikipedia:

Identifying Patterns in Editing and Article Creation. Is there a method to the madness of article creation and edits on Wikipedia?

Dataset: http://en.wikipedia.org/wiki/Wikipedia:Database_download

6) The Internet CD Database

Patterns in CD Releases (artists, releases, tracks etc.)

Patterns in User Generated Content Creation

Datasets: http://www.freedb.org/en/download__database.10.html

http://musicbrainz.org/doc/Database

7 ) IIMC Course Selection:

Is there a Pattern in the Selection of Courses across batches? What inferences can be drawn about type and behaviour of students?

Dataset: Extract from PGP Office

8 ) Course Selection, CGPA and Career:

How does Course Selection affect CGPA ? Does CGPA have a bearing on placement, career, success?

Dataset: Extract from PGP Office, Alumni Cell

9 ) Library Data Mining

Patterns in book issuance. Build a predictive model for future issuance.

Dataset: Contact IIMC Library

10) IPmsger

Simple Frequency Analysis. Co-Occurence of certain Words.

Text Mining of Log: Is there method to the IPmsg madness? Find Patterns and Insights on Campus Chat.

Dataset: IPMsger LogFile of past years.

You can use log-analyzers . See http://www.hypernews.org/HyperNews/get/www/log-analyzers.html

11) Entrepreneurship

What combination of factors lead to an entrepreneurial culture?

2007 World Bank Group Entrepreneurship Survey measures entrepreneurial activity in 84 developing and industrial countries over the period 2003-2005.

Dataset: http://www.ifc.org/ifcext/sme.nsf/Content/Entrepreneurship+Database

12) Financial Services

Finance for All? Find insights on Policies and Pitfalls in Expanding Access for the benefit of the many.

Dataset: http://econ.worldbank.org/WBSITE/EXTERNAL/EXTDEC/EXTRESEARCH/0,,contentMDK:21546633~pagePK:64214825~piPK:64214943~theSitePK:469382,00.html

13 ) Financial Structure

Construction of financial structure indicators to measure whether a country's banks are larger, more active, and more efficient than its stock markets. These indicators can then be used to investigate the empirical link between the legal, regulatory, and policy environment and indicators of financial structure. They can also be used to analyze the implications of financial structure for economic growth.

Dataset: http://econ.worldbank.org/WBSITE/EXTERNAL/EXTDEC/EXTRESEARCH/0,,contentMDK:20696167~pagePK:64214825~piPK:64214943~theSitePK:469382,00.html

14) Bank Regulation and Supervision

Analysis of the impact of bank regulation on various dimensions of bank performance. Study of factors that determines the decisions countries make on the orientation of the regulatory environment, and draw policy conclusions.

Dataset: http://econ.worldbank.org/WBSITE/EXTERNAL/EXTDEC/EXTRESEARCH/0,,contentMDK:20345037~pagePK:64214825~piPK:64214943~theSitePK:469382,00.html

15) Economic Growth and Environmental Quality

Analysis of Linkages between growth and environment quality

Dataset: http://econ.worldbank.org/WBSITE/EXTERNAL/EXTDEC/EXTRESEARCH/0,,contentMDK:20699819~pagePK:64214825~piPK:64214943~theSitePK:469382,00.html

16) Commonalities in Controversial Pages

Can you find patterns in Most Frequently Edited Pages on Wikipedia over time?

What is the relationship between Page Views and Page Edits on Wikipedia?

Dataset: http://en.wikipedia.org/wiki/Wikipedia:Most_frequently_edited_pages

17) Small States, Small Problems?

Patterns in different Nations's Problems and relations with Size.

Dataset: http://econ.worldbank.org/WBSITE/EXTERNAL/EXTDEC/EXTRESEARCH/0,,contentMDK:20699094~pagePK:64214825~piPK:64214943~theSitePK:469382,00.html

18) Data Mining of Electricity Regulation Dataset

How do the different variables in electricity regulatory governance impact performance?

Dataset: http://econ.worldbank.org/WBSITE/EXTERNAL/EXTDEC/EXTRESEARCH/0,,contentMDK:20699165~pagePK:64214825~piPK:64214943~theSitePK:469382,00.html

19 ) Fiscal Policy and Economic Growth

Investigating Interelationships between the two.

Dataset: http://econ.worldbank.org/WBSITE/EXTERNAL/EXTDEC/EXTRESEARCH/0,,contentMDK:20699114~pagePK:64214825~piPK:64214943~theSitePK:469382,00.html

20) Data Mining in Sports

Does past performance predict future preformance? Using statistical data about the past, is it possible to build a model of predictive value?

Dataset: Play Football Manager 2008, use the SAV file created mid-season and predict the rest of the season!

21) Facebook

Interaction Patterns in Online Social Networks

Dataset: Dummy Data can be got at : http://developers.facebook.com/fbopen/

22) Retail

Patterns in Buyer Behaviour.

Dataset: Get Data from Pantaloon, Big Bazaar, Monginis, etc. They would probably be willing, if sensitive information is blacked out.

23) Stock Markets

Are Closing Values telling us something valuable? With Cluster Analysis find out stocks that move together. Warning: Successful Completion of this Project could cause you to become a gazillionaire and risk dropping out of the course.

Dataset: Dig around for SENSEX/NIFTY backtesting data.

24) Vandal Detection

Wikipedia accepts edits even from anonymous editors. Can you device a model to identify the Vandal Edits automatically? (Have a look at http://www.research.ibm.com/visual/projects/history_flow/ for ideas)

Dataset: http://en.wikipedia.org/wiki/Wikipedia:Database_download

25) Website Optimization

Experiment with methods for predicting the next Web page a user will access

Dataset: Log Data from any website administrator. One possible source could be ISG.

26) News Mining

Mining Live News Data Streams for Patterns

Dataset: Any of the numerous news feeds.

27) Seeker behaviour on the Internet

Is there a pattern to the topics sought out by seekers f information on the Internet?

Dataset: Search Volumes Data from compete.com, alexa, etc. Detailed data from WikiStats about page views on Wikipedia: http://dammit.lt/wikistats/

28) The Role of Macroeconomic Factors in Growth

Is growth ireally negatively associated with inflation, large budget deficits, and

distorted foreign exchange markets? Investigations and Insights required!

Dataset: http://econ.worldbank.org/WBSITE/EXTERNAL/EXTDEC/EXTRESEARCH/0,,contentMDK:20699104~pagePK:64214825~piPK:64214943~theSitePK:469382,00.html

29) Usage of UNIX

What are the trends in the way people use UNIX?

Dataset: http://pages.cpsc.ucalgary.ca/~saul/wiki/pmwiki.php/Resources/DataSets

30) Usage of Web Browsers

How do users use web browsers? Are there identifyable patterns across clusters of users?

Dataset: http://pages.cpsc.ucalgary.ca/~saul/wiki/pmwiki.php/Resources/DataSets

31) Health Risk Analysis of Adolescents in India

---- Live Project ------

What are the major health risks facing adolescents? What aspects of their psychographic, demographic, socio-cultural-economic factors can they be traced back to? Are there any trends visible?

Dataset: Survey in progress at a Hospital in Mumbai, Data available on demand.

Feel free to dig around for more. Interesting data is lying around in the unlikeliest places :)

You can start off by having a look here: http://kdd.ics.uci.edu/

A very nice reference is to be found here:

http://delicious.com/pskomoroch/dataset

http://www.datawrangling.com/some-datasets-available-on-the-web

Explore the datasets available.

Think. Ponder. Mull.

Gaze intensely into the distance, hand on chin.

Make sure you have a solid Rationale for what you plan to do, in your Project Proposal Document.

Welcome to BI Tools and Techniques

2008-09-07T23:25:00.000-07:00

This blog has been created to provide assistance as well as an ongoing record of the Projects undertaken as part of the Business Intelligence Tools and Techniques Course at IIM Calcutta. The course co-coordinators are Prof Asim Pal and Prof Ambuj Mahanti.

I am Myshkin Ingawale, your Teaching Assistant for this course. You may fire all your project related queries at me, and thereby give me the opportunity to either display my profound ignorance or more hopefully, guide you along the path to Business Intelligence Awesomeness.

While, generally speaking, anything goes as far as type, quantity and quality of posts on this blog is concerned, you are requested to follow the following guidelines:

Respect other people's posts: Do not Edit any posts bar your own.
Please make sure you have something substantial to say either about your project or about the subject in general before you initiate a new post here. Short conversations and requests for info (on deadlines, quizzes, tools for Total World Domination, etc) are for the BB or the Comments section.
While posting Project related information (such as Project Proposal) , please Title your post by the name and number of your project, team number and submission type. E.g: Financial Crisis | Project No xx | Team No xx | Project Proposal
As your project proceeds along its way, feel free to add matter to your existing project post rather than creating a brand new post for every new change made to the project.
Data Redundancy makes for Reliability: Keep a Doc copy of important submissions like Project Proposal, etc on your harddrive.