Guide: Finding Matrix Diagonals


As part of my “Get Ready for Senior Tech Interviews” I was looking at Hacker Rank Questions and parsing through diagnoals of a Matrix Came up.

Obviously the “I am at work, not in an interview way is to use numpy, diag” However this it about understanding the iterations instead

Guide to Finding Matrix Diagonals <!– HOW TO USE IN WORDPRESS: 1. Edit a page or post in WordPress. 2. Add a "Custom HTML" block where you want this guide to appear. 3. Copy everything from the tag below to the closing tag… 4. …and paste it directly into the Custom HTML block. 5. The guide should now render correctly on your WordPress site. –> https://cdn.tailwindcss.com body { font-family: ‘Inter’, sans-serif; } .matrix-grid { display: grid; gap: 0.5rem; } .matrix-grid.grid-3 { grid-template-columns: repeat(3, minmax(0, 1fr)); } .matrix-grid.grid-4 { grid-template-columns: repeat(4, minmax(0, 1fr)); } .matrix-cell { width: 4rem; /* 64px */ height: 4rem; /* 64px */ display: flex; align-items: center; justify-content: center; font-size: 1.25rem; /* 20px */ font-weight: 700; border-radius: 0.5rem; /* 8px */ background-color: rgb(229 231 235); color: rgb(31 41 55); transition-property: all; transition-timing-function: cubic-bezier(0.4, 0, 0.2, 1); transition-duration: 300ms; } .highlight-path { background-color: rgb(16 185 129); color: #fff; box-shadow: 0 10px 15px -3px rgb(0 0 0 / 0.1), 0 4px 6px -4px rgb(0 0 0 / 0.1); transform: scale(1.1); } .highlight-path-main { background-color: rgb(99 102 241); color: #fff; box-shadow: 0 10px 15px -3px rgb(0 0 0 / 0.1), 0 4px 6px -4px rgb(0 0 0 / 0.1); transform: scale(1.1); } .highlight-row { /* Using border instead of ring for better compatibility */ border: 4px solid rgb(56 189 248); } .highlight-col { /* Using border instead of ring for better compatibility */ border: 4px solid rgb(236 72 153); } .code-block { background-color: rgb(17 24 39); color: #fff; padding: 1rem; border-radius: 0.5rem; font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, “Liberation Mono”, “Courier New”, monospace; font-size: 0.875rem; /* 14px */ overflow-x: auto; }

Finding Matrix Diagonals

An Illustrated Guide to the Python Techniques

1. The Main Diagonal (Top-Left to Bottom-Right)

The Core Rule

For the main diagonal, the rule is the simplest. An element is on the main diagonal if its row index is equal to its column index.

row_index == column_index

The Python Code

# For the main diagonal, we use enumerate to get the index 'i'
main_diagonal = [row[i] for i, row in enumerate(matrix)]

# Or, more simply with a range
main_diagonal = [matrix[i][i] for i in range(n)]

# For matrix = [[10,20,30],[40,50,60],[70,80,90]], result is [10, 50, 90]

Interactive Example: Main Diagonal

Click through the steps. Here, n = 3 and we look for elements where row_index == col_index.

Next Step Reset

Resulting List:

[ ]

2. The Anti-Diagonal (Top-Right to Bottom-Left)

The Core Rule

For the anti-diagonal, the sum of an element’s row index and column index is always equal to n - 1.

row_index + column_index = n – 1

The Python Code

# For the anti-diagonal, we use the formula: col = n - 1 - row
anti_diagonal = [matrix[i][n - 1 - i] for i in range(n)]

# For matrix = [[10,20,30],[40,50,60],[70,80,90]], result is [30, 50, 70]

Interactive Example: Anti-Diagonal

Click through the steps. Here, n = 3, so the target index sum is n - 1 = 2.

Next Step Reset

Resulting List:

[ ]
const matrixData = [ [10, 20, 30], [40, 50, 60], [70, 80, 90] ]; const n = matrixData.length; function setupInteractiveExample(config) { let currentStep = -1; const result = []; const matrixContainer = document.getElementById(config.matrixId); const stepInfoContainer = document.getElementById(config.stepInfoId); const resultList = document.getElementById(config.resultListId); const nextStepBtn = document.getElementById(config.nextBtnId); const resetBtn = document.getElementById(config.resetBtnId); function createMatrix() { matrixContainer.innerHTML = ”; matrixData.flat().forEach((val, index) => { const cell = document.createElement(‘div’); cell.className = ‘matrix-cell’; cell.id = `${config.prefix}-cell-${Math.floor(index / n)}-${index % n}`; cell.textContent = val; matrixContainer.appendChild(cell); }); } function resetHighlights() { matrixContainer.querySelectorAll(‘.matrix-cell’).forEach(cell => { cell.classList.remove(config.highlightClass, ‘highlight-row’, ‘highlight-col’); cell.style.border = ‘none’; // Reset border style }); } function updateStep() { resetHighlights(); if (currentStep >= n) { stepInfoContainer.innerHTML = `

Finished!

The loop is complete and the final list has been generated.

`; nextStepBtn.disabled = true; nextStepBtn.classList.add(‘opacity-50’, ‘cursor-not-allowed’); return; } const { i, j, value } = config.logic(currentStep); result.push(value); for (let k = 0; k { currentStep++; updateStep(); }); resetBtn.addEventListener(‘click’, () => { currentStep = -1; result.length = 0; resetHighlights(); stepInfoContainer.innerHTML = `

Step 0: Start

Click “Next Step” to begin the loop.

`; resultList.textContent = ‘[ ]’; nextStepBtn.disabled = false; nextStepBtn.classList.remove(‘opacity-50’, ‘cursor-not-allowed’); }); createMatrix(); resetBtn.click(); } // — Configuration for Main Diagonal — setupInteractiveExample({ prefix: ‘main’, matrixId: ‘main-interactive-matrix’, stepInfoId: ‘main-step-info’, resultListId: ‘main-result-list’, nextBtnId: ‘main-next-step-btn’, resetBtnId: ‘main-reset-btn’, highlightClass: ‘highlight-path-main’, logic: (step) => ({ i: step, j: step, value: matrixData[step][step] }), explanation: (i, j, value) => `

Step ${i + 1}: (i = ${i})

The rule is row == col, so we get the element at matrix[${i}][${i}].

Value: ${value}

` }); // — Configuration for Anti-Diagonal — setupInteractiveExample({ prefix: ‘anti’, matrixId: ‘anti-interactive-matrix’, stepInfoId: ‘anti-step-info’, resultListId: ‘anti-result-list’, nextBtnId: ‘anti-next-step-btn’, resetBtnId: ‘anti-reset-btn’, highlightClass: ‘highlight-path’, logic: (step) => ({ i: step, j: n – 1 – step, value: matrixData[step][n – 1 – step] }), explanation: (i, j, value) => `

Step ${i + 1}: (i = ${i})

Calculate column: j = n-1-i => ${n}-1-${i} => ${j}.

Coordinates: matrix[${i}][${j}]

Value: ${value}

` });

Google Takeout – Music Activity Data


After looking through the Takeout data for the tracks I have listened to I cannot see any sort of unique ref for track, artist or album. There isn’t any id on the .csv file itself. Uh oh. I realised this was super gross. None of the data would be linked to the actual track ids etc.

Photo by Martijn Baudoin on Unsplash

This is fine for skilling up for some Data Science techniques, but otherwise? Unless I am getting something seriously wrong here Google have really dropped the ball on interlinking their data here… I mean, its GOOGLE right?

I scratched around and found that a sub section of Google Activity would help on the when, and applied for the archive. Surely this would be better and all would be revealed?

lets deselect all the other guff right now as that is pointless work to wade through later for a few clicks now

At first I got the wrong format, as I intended to use pandas.DataFrame functions so I’d be better off in JSON for the starting point.

when working with pandas JSON is much much easier than a mess of HTML

Lets have a look at this JSON via a Jupyter Notebook display on head and see what we can determine

import requests
import json
import pandas as pd
from pandas.io.json import json_normalize 
r = requests.get(r'https://raw.githubusercontent.com/FinancialRADDeveloper/GoogleMusicDataAnalysis/master/takeout_data/My%20Activity/Google%20Play%20Music/My%20Activity.json')
activity_data_frame = json_normalize(r.json())

display(activity_data_frame.head())

This isn’t good news either. WTAF Google, how is anyone supposed to use this data?

WTAFactivity dataframe head()

Ok, so it is just as bad for interlinking as it was for track data. I’m kicking myself for not using Spotify premium a few years ago! Maybe I’m missing something but the Spotofy API is littered with URIs and other data to link everything together, whereas with Google Music I’m left with the option scrabbling around with text parsing? However, while annoying, it is perfect for skilling up on some data science basics.

This leaves some questions we need to answer:

  • Can we do data verification between the tracks listened to, and the Google Music activity?
  • What does the general total and daily activity look like?
    • How should I display this?
  • Can I display meaning ful stats on most popular choice per hour of the day?

I think any further analysis will start to need to delve into some more Meta data, which will be covered in another post regarding web scraping or Spofy API via spotipy.

Listening stats per hour

import matplotlib.pyplot as plt

plt.ion()
plt.bar([i[0] for i in daily_listening_stats], [j[1] for j in daily_listening_stats], color = 'blue')
plt.bar([i[0] for i in skipped_stats], [j[1] for j in skipped_stats], color='red')
plt.bar([i[0] for i in search_stats], [j[1] for j in search_stats], color='purple')
plt.xlabel('Hour of the Day')
plt.ylabel('Total tracks')

plt.show()

This seems roughly in line with expectations. Some late at night then a big dip while asleep and then a ramp up. I do’t tend to listen much on the commute, but will listen to something at the start of the working day. Then a dip for lunch, and some listening from then till tailing down on the eveing when eating or watching TV with my wife.

Daily Listening Stats:

import requests
import json
import pandas as pd
from pandas.io.json import json_normalize 
import matplotlib.pyplot as plt


r = requests.get(r'https://raw.githubusercontent.com/FinancialRADDeveloper/GoogleMusicDataAnalysis/master/takeout_data/My%20Activity/Google%20Play%20Music/My%20Activity.json')
activity_data_frame = json_normalize(r.json())

listened_to_dataframe = activity_data_frame[activity_data_frame['title'].str.contains("Listened to")]

display(listened_to_dataframe.sample(10))

listened_to_dataframe['activity_datetime'] = pd.to_datetime(listened_to_dataframe['time']).copy()

daily_stats = listened_to_dataframe['activity_datetime'].dt.date.value_counts()
daily_stats_sorted = daily_stats.sort_values(ascending=True)
daily_stats_sorted.plot(legend=True)

Further analysis and visualisation

It would be really interesting to get into the meta data.. I can work out my favourite tracks and artists per hour of the day but its going to be much harder to work out styles or genres. As mentioned before it should force me to go and learn another area of Data Science and python though.

Cross Validation

If each track entry has a ‘Play Count’ field, can I safely assume that if i pick out a sample of tracks, I’ll be able to find the correct Activity entries? Let us see shall we. I’ll take some of the most popular tracks, as a few missed listens should matter alot less.

Foray Into Data Science – Part II


As mentioned in my earlier article, I am starting to get into Data Science. I might be quite late to the party in some respects but better late then never I guess.

https://financialraddeveloper.com/2019/09/01/initial-foray-into-data-science-with-python/

Building from my intial thoughts and ideas I thought I would go throug the process, as I experienced it, in a series of posts here. I often read highly polished “how to” articles but I would like to include a warts and all journey through all the obstacles / sticking points and general quagmire of head scratching and confusion that people (maybe it is just me) experience when starting out something new.

To be honest even restarting / setting up this blog after a significant haitus was a > 1hr chore I wasn’t expecting. Even then I am not sure if the WordPress.exe is worth the time or RAM it sucks in.

Equipment:

This shaped quite a bit of my decisions later on down the line so I will state what I have, so that those calls don’t look so odd later on.

The ASUS Powerbook tablet combo was a few hundred quid back in 2015 but my wife didn’t like it, and so I dusted it off after a colleague intrigued me with his much fancier Windows Surface Pro style laptop. For the < £150 that it can be purchased for now it is a total bargain, especially with the ability to put a memory card in the tablet part and having a 500Gb HDD in the keyboard. As I reckon I cannot realistically develop any meaningful Python without a half decent keyboard ( and certainly not an on screen one ) I installed the PyCharm and Anaconda parts to the E: drive that comes up when the unit is “complete”.

One nice side effect is that I have to really consider performance and efficiency when I am both developing and also in my own environment. Which explains my early doors switch away from PyCharm and into Spyder and Jupyter Notebooks. The overhead of the Java based Eclipse part of PyCharm caused my morning train development to really slow to a crawl.

On a side note I got a great deal on my HP Z600 Workstation which until my clients switch to Win 10 a clone of my professional development workstation

This will be reclaimed old stock from somewhere but I love a bargain and the idea of getting that much RAM and CPU for under £500 when some company probably specced that bad boy 5+ years ago for £5k makes me very happy.

Initial foray into Data Science with Python


Professionally I have undertaken a major segue into Python from the usual Excel VBA development environment.

However this has remained very tightly linked to the quant library and specific business requirement solving as before.   So Iface the intresteing situation of nominally becoming a “Python Developer” without having the necessary interview answers or expected jobskills.

What I mean is that with Excel VBA work within Finance, and especially supporting the desks of Wholesale banks certain things are expected.   Working with Front Office designed sheets and integrating those and also the desk requirements into solutions using the IT designated strategic frameworks and technology.

So while you are expected to be very experienced / capable in Excel VBA itself it is understood that this will be back up with a large amount of esoteric and niche software across the “tech stack”.   Mainly IT developed XLAs / sheet utils and other “best practice” while using a normally mature (and highly guarded IP) layer of Quant library in some other language such as C++ / Java.

So getting things done is a case of elegantly adding in the solution to the spaghetti code thatis already there, without reinventing the wheel and / or making it any worse than it already is.    A majority of the heavy lifting on the data manipulation, trade access, market retrieval and manipulation,  valuation functionality will already be implemented bythe quant / strats function.

However the open nature of the python distribution and usage paradigm would make it insane for a bank to re invent / ignore numpy, pandas, scikit etc.   In my opinion not using the basic python libs available in Anaconda is madness.

So this leaves me with the strange situation that I am using python coupled with the quantlibrary to solve complex business problems without actually really using much of the available libraries themselves.

In effect I will be a 2 year Python developer ( on top of 12 – 15 y of financial software development ) without really being able to back that up.  Unless discussing highly proprietary Variable Annuity cases in any future project / interview.

Anyone who knows large Investment / Wholesale banking IT will know that picking up and running with some new ( to the bank ) technology and thinking isnt the done thing.  It is a configuration nightmare and makes a lot of senior people in both Front Office and also IT quite nervous.

New is bad, things should creep in slowly.   Well… Python has crept in and our small teamhas a chance to utilise most of its power as we see fit.   So I plan to familiarise myself with the Data Science elements via extra curricular development and hope that keeps my CV sharp while also providing a chance to try something interesting.

Quick Transposer


A post from a Pistonhead member in need :

Hopefully someone can help!

I have a list of approx 50000 numbers in ‘Column A’ of spreadsheet. I need to split this list into batches of 1000, populated into Column B onwards.

Anyone with some VBA skill got any ideas? Thanks in advance!

 
My reply :
 
Sub QuickTranspose()

    Dim rng As Range
    Dim StartRowNum As Long
    Dim EndRowNum As Long
    Dim ColumnOffsetCounter As Long
    Dim GroupSize As Long
    Dim v
    Dim i As Long
    Dim TotalRows As Long
    GroupSize = 1000
    TotalRows = 50000
    
    'Dirty but without range names etc you just need to get the
    'selected cell to be the first one in the list
    Set rng = Application.Selection
    StartRowNum = rng.Rows.Row
    For i = 0 To TotalRows / GroupSize
        EndRowNum = StartRowNum + GroupSize - 1
        v = Range("A" & StartRowNum & ":A" & EndRowNum).Value
        Range(rng.Offset(0, i + 1), rng.Offset(GroupSize - 1, i + 1)).Value = v
        StartRowNum = EndRowNum + 1
    
    Next i
End Sub

Watch Accuracy Tracker


I had an idea the other day that I would write a small app in Excel to keep track of all my watches.

I was planning to use a constant source like :

http://time.is/

to keep track of the ‘real’ time and then have an ability to take a ‘snapshot’ of what the time is on each watch in a list ( that can be added to of course). This way using the inbuilt goodies of Excel all manner of graphs and stats can be applied to the raw data and people can see how their watch is behaving.

I was also thinking of adding a few features like resting orientation to see if any of the theories about leaving the watch 3 down, 9 down , dial up, dial down do influence the readings. Along with whether the chrono is running, is it being worn during the timing period etc.

Would anyone be interested in a copy of this? If anyone has any ideas / suggestions that would be great.

I am also thinking of using this as a quide too :

http://rcm-uk.amazon.co.uk/e/cm?lt1=_blank&bc1=000000&IS2=1&bg1=FFFFFF&fc1=000000&lc1=0000FF&t=finraddev-21&o=2&p=8&l=as4&m=amazon&f=ifr&asins=B004BLIW22

G Challenge II – Roman Numerals


Ok, in quick sucession here is another G Challenge.  I think this one might be a bit of a stinker though!

II  (2) ;p   Functions that can be called from Excel to convert a number to its Roman equivalent, and also back from Roman to Normal.

So :

2010 -> MMX

Too easy… ?

231241 -> ccxxxMCCXLI

I have yet to attempt this myself as I normally set challenges from something I have had to do in the past I have found tricky.  This came from me choosing to put the challenge number as II not 2, and it popped into my head!

Edit:  I *know* that ROMAN() exists but that only goes up to 3999, and I am interpreting this as ‘classic’ numerals.    I think having looked at it that this is a massive challenge for a learner, but I have posted it now and we can always come back to it.   I’ll need something more sympathetic for GC III

G Challenge I


Ok, my friend is trying to learn Excel VBA and get into RAD development.   I am going to set a number of challenges which I am going to call a ‘G Challenge’  😛

So the first.  Take a string in the format :

20101124_170615_dd20101124_EOD.g

where the format is :

yyyymmdd_hhmmss_ddyyyymmdd_EOD.g

In VBA I would like 2 things. 

  1. The string back in the format dd-mmm-yy hh:mm
  2. An actual date serial

Example above would go to 

  1. 24-Nov-2010 17:06:15
  2. 40506.7126736111

Go!   I would use 2 seperate functions to start off with.

Answers will be forthcoming in a post G Answer I

New Financial Product Today – PRD & Callable PRD


After all my interviewing ( more on that to come ) I have a new contract. As such, a new bank and a new lot of things to learn. So while watching a training video I cam across a mention of PRD’s and callable PRD’s. I just had to look it up. Google didn’t yield as much, but trusty ol’ Wikipedia did the job as per usual.

Power Reverse Dual – there we go.

http://en.wikipedia.org/wiki/Power_reverse_dual_currency_note

http://giddy.org/sf/articles/structured_notes.pdf

Further Excel VBA RAD Interview Questions


I had an interview today.  I didn’t do all that well at all and I got a little bit schooled.

This always seems to happen on my first technical interview with any serious technical ‘harrasment’.  I also have a load of knowledge tucked away in my brain, none of which the guy wanted to know about sadly.

Here are the things he did want to know about that I really got flustered and couldn’t answer.  Embarrasingly I gave myself an 8 out of 10, when I actually performed like a 5.5.  I think he rated himself as an 8, and then quite easily proceeded to destroy me. 😦  boo.

So… back to the hard core technical stuff for me as I spent too long this week on the finance side and not enough on the actual RAD side, silly me.

Ranges:  How do you create a range that will grow as and when data is enetered into it.

  • Name the 5 types of join on a database
  • how can I get a range to grow when data is added ( VBA & Excel)
  • How can I change the Last Used cell in Excel.
  • C++ pointers
  • Basic C# questions.

Ooh, and make sure that you can eloquently discuss the previous projects you have had on your CV.  Especially be prepared to get right into the nitty gritty.  I was silly enough to let information like the usings and data types etc I was using in one of my projects I only stopped working on about 8 months ago.  Not great either.

If I have a column with data in it.

ab

cd

ef

and I want to add in another item of data, but have the range grow.  What can I do.

5 Types of Join – I was going to write the answer in here, but now I realise that what I need to do… is have a post dediected to SQL.  Any decent RAD developer is going to have to dig around in a database from time to time.  I think the problem I had is that I let the info go too stale, and didn’t keep on top of it.  I mean sure I seriously believe 100% I could do the job.  I would merely look in w3schools / Wikipedia for a few minutes and then be away and fine.  However… interviews just aren’t like that.  You need a decent answer, confident and concise right there, at the time to give the person that warm fuzzy glow that gets one hired.

Snippet :

Here are the first few things I selected to look at.   A simple glance over these for 30-90 mins before the interview would  have given me a massive boost in the interview:

http://www.w3schools.com/sql/sql_join.asp

http://www.tomjewett.com/dbdesign/dbdesign.php?page=jointypes.php

http://en.wikipedia.org/wiki/Join_%28SQL%29

Summary ( for my own revision )

Dynamic Ranges:

http://www.ozgrid.com/Excel/DynamicRanges.htm

Last Used Cell:

Run a small macro.. ActiveSheet.UsedRange.   Or you can do this in the Immediate window if you don’t want it to be anywhere in the code.

Blog at WordPress.com.

Up ↑